Category Archives: JSON-RPC

All JSON-RPC 2.0 libraries are now in Maven Central

masthead-jsonrpc2baseThe migration from Ant to Maven is now complete and all Java libraries for handling JSON-RPC 2.0 messages are now published in Maven Central.

You can find them under the com.thetransactioncompany.com groupId:

The useful JSON-RPC 2.0 Shell tool, available for purchase, has also been completely mavenised. It has found a lot of use in Android development recently and there have been plans to add HTTP basic authentication support to it (via a prompt).

JSON-RPC 2.0 libraries and tools switch to Maven

masthead-jsonrpc2baseThe Java libraries and tools for JSON-RPC 2.0 message serving and processing now use Apache Maven to build.

Migrating the previous Apache Ant scripts took about a day a to complete and was not without hassles, notable the ZIP package distribution and the automatic versioning of JavaDocs. We’ll now be able to gain from Maven’s automatic dependency resolution and I have gained enough knowledge to begin considering myself a Maven power user 🙂

The following related JSON-RPC 2.0 projects also got a Maven make-over:

Publishing to Maven Central is on the to-do list complete now 🙂

The JSON-RPC 2.0 Client can now handle GZIP and DEFLATE compression

Does your JSON-RPC server return large amounts of data? If so applying HTTP response compression can greatly reduce network traffic and response time, occasionally by a whole magnitude if your JSON structures contain long repeating keys or values.

Today we released a new version of the JSON-RPC 2.0 Client library for Java that adds support for GZIP and DEFLATE compression. These two algorithms are commonly implemented by web servers such as Apache HTTPD and Apache Tomcat.

To enable HTTP response compression in your Java client app:

URL serverURL = new URL("http://example.com/jsonrpc2/");

JSONRPC2Session mySession = new JSONRPC2Session(serverURL);

mySession.getOptions().enableCompression(true);

The JSON-RPC client will then set the Accept-Encoding request header to signal that it can process compressed responses. Note that many web servers don’t have HTTP compression enabled by default so this setting will simply be ignored by them.

To enable compression in Apache Tomcat you need to configure the HTTP connector by setting compression to “on”.

The JSON-RPC client will then automatically decompress the HTTP responses before processing the JSON payload.

The new version of the JSON-RPC client library also improves HTTP cookie support. It has dropped the primitive internal cookie store in favour of the standard java.net.CookieManager so now cookie expiration and replacement are also handled.

Access filters for JSON-RPC 2.0 services

Services on the web often have to be controlled for who and how accesses them. If they deal with sensitive data, such as usernames and passwords, you may want to ensure that all requests come in over encrypted HTTPS. If you’re running a private service, you may want to ensure that only selected internet hosts can make use of it. Or you may want to limit access to portions of the web API, such as administration, to selected users and clients.

For this purpose I created an Access Filter package for JSON-RPC 2.0 services. Today I released it to the public in its stable 1.1 version under a generous Apache 2.0 licence. I wanted filtering to be fast and robust, so the package was designed to be as simple and efficient as possible. It also nicely complements the proven Java libraries JSON-RPC 2.0 Base and JSON-RPC 2.0 Server.

The package defines four basic filters:

  • HostFilter Access filter ensuring JSON-RPC 2.0 requests originate from selected host names / IP addresses.
  • HTTPSFilter Access filter ensuring requests are passed over HTTPS.
  • X509ClientCertFilter Access filter ensuring HTTPS requests carry an X.509 client certificate with optional specified principal (subject DN).
  • APIKeyFilter Access filter ensuring clients present an API key for protected JSON-RPC 2.0 request methods.

There is a ready CompositeFitler which combines these four basic filters into a chain. You can then plug this filter chain into your JSON-RPC 2.0 service and call it prior to passing the incoming requests to the actual execution logic.

The CompositeFilter can be configured with a simple properties text file. The properties format is detailed in the JavaDocs for the CompositeFilterConfiguration class.

access.https.require=true
access.https.requireClientCert=true
access.https.clientCertPrincipal=cn=John Doe,ou=people,cd=company,dc=org
 
access.hosts.allow=*
 
access.apiKeys.require=true
access.apiKeys.exemptedMethods=ws.getName ws.getVersion ws.getTime
access.apiKeys.map.f70defbe-b881-41f8-8138-bea52b6e1b9c=sso.login sso.logout sso.getSession
access.apiKeys.map.08d1e641-b1c1-4d88-8796-e47c06430efb=sso.proxiedLogin sso.proxiedLogout sso.getSession
access.apiKeys.map.d881afe0-4d7d-4520-9fda-bffffc3022ba=sso.userCount sso.sessionCount sso.listUsers

Alternatively, you may devise your own custom filter chain, based on the AccessFilterChain class.

How do you use the CompositeFitler?

// Parse config from properties, create composite filter
java.util.Properties props = ...
CompositeFilter filter = new CompositeFilter();
filter.init(new CompositeFilterConfiguration(props)); 

// Invoke the filter for each incoming JSON-RPC 2.0 request

// Parse JSON-RPC 2.0 request
JSONRPC2Request request = JSONRPC2Request.parse(...);

// Set the message context, i.e. client host IP, etc.
MessageContext mctx = new MessageContext(...);

// Call the filter
AccessFilterResult result = filter.filter(request, mctx);

if (result.accessAllowed()) {
    // Continue with request processing...

}
else {
    // Return specified access denied JSON-RPC 2.0 error
    // to the calling client
    JSONRPC2Error error = result.getAccessDeniedError().toJSONRPC2Error();
}

The Access Filter package is available at Bitbucket: https://bitbucket.org/vdzhuvinov/access-filter/

We plan to integrate into the upcoming new releases of the Json2Ldap, NimbusSSO and AuthService web services from NimbusDS software.

If you have comments of feedback to share, get in touch or leave a ticket at the Bitbucket repo.

Plain SASL authentication

Json2Ldap iconYesterday’s 2.0 release of Json2Ldap brings a lot of new good things. Some of them are hidden, representing various little stubs under the hood that will enable cool new features to be added in future (patience, you’ll find out in due time!). On the outside, the most noticeable addition is the arrival of plain SASL authentication support. This LDAP authentication method offers two key advantages over the traditional simple bind. Here I’ll explain what these are and how you could apply them.

Simpler authentication with username instead of DN

The traditional ldap.simpleBind method authenticates users with their distinguished name (DN) and password. Using DNs for web app login, however, is a bit cumbersome. Why? Because you first have to establish a search connection (bound as some service user) in order to resolve the user’s DN from the entered username or email address, typically with a filter like "|(uid=%u)(mail=%u)", and only then you can make the actual ldap.simpleBind request.

ldap.plainBind spares us these preliminary steps by taking care of the underlying user DN resolution. With it you can just pass the user’s name / email + password and be authenticated. The only snag is that the LDAP directory may require some configuration before that, to allow plain SASL binds and to know which user record attributes will serve as “username”.

Here is an example plain SASL auth included in the ldap.connect request:

{ "method"  : "ldap.connect",
  "params"  : { "host"      : "ds.wonderland.net",
                "port"      : 389,
	        "security"  : "StartTLS",
		"plainBind" : { "username" : "alice",
		                "password" : "secret" },
  "id"      : 1,
  "jsonrpc" : "2.0" }

Proxied authorisation / switching the current user

Proxied authorisation is similar to the sudo command on Unix systems. It allows you to authenticate to the directory with the credentials of one user and then perform all subsequent operations as another user. This is the other significant feature of plain SASL bind.

{ "method"  : "ldap.connect",
  "params"  : { "host"      : "ds.wonderland.net",
                "port"      : 389,
	        "security"  : "StartTLS",
		"plainBind" : { "username"       : "myWebApp",
		                "targetUsername" : "alice",
		                "password"       : "secret" },
  "id"      : 1,
  "jsonrpc" : "2.0" }

Where could proxied authorisation come to use?

In situations when a web app has to modify user data on behalf of another user. In the above example the JSON call authenticates as “myWebApp” but then requests the resulting connection to receive the identity (and privileges) of user “alice”.

Proxied authorisation can also be used to create custom external login procedures, for example based on OAuth 2.0 tokens. Here a helper app upon establishing the identity of the user through some other external mean authenticates to the directory as itself but then through the targetDN / targetUsername parameter requests the connection to take on the identity of the target user.

Note that enabling proxied authorisation for a particular service account requires the underlying LDAP directory to be explicitly configured for that. Check out the docs of your particular directory server for detailed instructions.

Which directory servers support PLAIN SASL bind (RFC 4616)?

  • OpenDJ / OpenDS
  • OpenLDAP
  • Apache DS
  • 389 Directory Server
  • UnboundID in-memory DS
  • Novell eDirectory
  • IBM Tivoli Directory Server

Directory servers without PLAIN SASL bind support:

  • MS Active Directory

JSON-RPC and cookies

JSON-RPC ShellToday’s releases of the JSON-RPC 2.0 Client (version 1.6) and JSON-RPC 2.0 Shell (version 1.10) include support for handling HTTP cookies, just like browsers do. These latest updates came about after a developer in Norway wrote that the JSON-RPC software was not working as expected against the web API of a particular service provider. A quick look at the API docs revealed that the service was pushing session tokens outside the regular JSON-RPC message flow, using HTTP cookies. The inclusion of cookie support took about a day to code and test out and is now publicly available.

I know, to purists using out-of-channel means to pass the session tokens of a JSON-RPC service is not a clean thing to do. I also value the simplicity of having all message data passed in-channel and our JSON-RPC based services – from Json2Ldap to JsonSSO use the regular JSON-RPC parameters and result fields to communicate session tokens. But at the same time, if you’re programming a servlet-based web service, using the built-in session mechanisms can be too convenient (and routinely done) to ignore.

There is another pitfall with passing cookies from a JSON-RPC service: if you want to allow cross-domain XHR access from browsers to your service, IE will certainly block any cookies for security reasons.

So my advice is to stay away from cookies in JSON-RPC, despite their relative convenience.

JSON-RPC for Android apps

JSON-RPC 2.0 Thanks to user feedback I recently discovered that the JSON-RPC 2.0 Base and related libraries and JSON-RPC shell are finding increasing use in Android applications, mostly to hook to remote JSON-RPC services.

While I’m not really into mobile gadgets and don’t own a smart phone, I do sense there is usefulness in mobile apps to people. I’m now considering a mobile version of the Json2Ldap demo. The upcoming Transaction Company software will also certainly a boast a few mobile apps, to allow users to check their balance and pay co-workers while on the go. Application Craft‘s offering may be just the right mean to do that, as it provides both a development and hosting environment in one.

Easy LDAP user authentication over the web

NimbusDS AuthServiceAuthService is a simple web service software offered by Nimbus Directory Services. It can authenticate users against any LDAP directory using a web-friendly HTTP+JSON protocol.

Where to use?

AuthService can be used in any situation where you have to authenticate user credentials (login) over the web against an LDAP-compatible directory such as Microsoft Active Directory, Novell eDirectory or OpenLDAP.

  • You’re a SaaS provider and want to provide your corporate customers with a method for authenticating users against their internal directory.
  • Your company has applications in the cloud and wish them to use the corporate directory for user login.
  • You’re designing an Ajax app and want to authenticate users with a single JSON XHR. No redirection and page reloading, please!

How does it work?

AuthService is a nimble web service that sits in front of an LDAP directory and receives JSON-RPC request from web clients. It resolves the supplied usernames to valid directory entry entries (DNs) and translates between the binary LDAP protocol and HTTP/JSON.

AuthService authenticates users against an LDAP-compatible directory

The JSON request and response messages, what do they look like?

AuthService communicates with simple JSON messages. These follow the standard JSON-RPC 2.0 protocol.

Example user authentication request:

{ 
  "method"  : "user.auth",
  "params"  : { "username" : "alice@wonder.net",
                "password" : "s3cr3t" },
  "id"      : "0001",
  "jsonrpc" : "2.0" 
}

And here is an example response indicating the supplied username and password were correct:

{ 
  "result"  : true,
  "id"      : "0001",
  "jsonrpc" : "2.0"
}

The complete AuthService web API is documented online.

Can you retrieve a user’s details from the directory?

Yes! In addition to checking the user’s credentials you can also configure AuthService to return a selected set of user’s directory entry attributes on auth success. This is done via the authService.userAttributes configuration setting.

Here is a sample list of attributes that can typically be retrieved:

  • Given name, surname, full name.
  • Email address.
  • Office, home and mobile phone numbers.
  • Organisational details such as manager, department and authorisations / permissions.

Here is an example of such a JSON user auth response that returns selected user details:

{ "result"  : { "userID" : "alice",
                "name"   : "Alice Wonderland",
		"email"  : "alice@wonderland.net",
		"phone"  : "+44 711 567 8910" }
  "id"      : "0001",
  "jsonrpc" : "2.0"
}

Which directory servers is AuthService compatible with?

AuthService works with any LDAP v3 compatible directory. This includes Microsoft Active Directory, Novell eDirectory, IBM’s and Oracle’s directory services, the open source OpenLDAP, OpenDJ and ApacheDS servers.

What about installation?

AuthService is delivered as a standard Java servlet application, packaged as a WAR file and ready for immediate installation into any Java web server, such as the popular free Apache Tomcat or Jetty servers. The WAR file weighs a nimble 2.5 megabytes.

How about security?

AuthService includes a number of configuration settings for managing and protecting access to the service.

  • Whitelist of allowed client IPs / hostnames that may access the service.
  • Whether secure HTTPS is mandatory.
  • Whether clients are required to authenticate with a valid X.509 certificate issued by a trusted certificate authority (CA).
  • Whether the subject of the presented X.509 client certificate must match an authorised distinct name (DN).

What about price and support?

AuthService licenses come at a super affordable price. For immediate purchase and download check out the NimbusDS online shop. Attractive discounts are available if you with to distribute the software as part of your own product or SaaS offerings, for 100, thousand or even more copies.

NimbusDS, which operates out of London, UK and Bulgaria, also offers excellent integration and consulting services, with emphasis on LDAP directory design, meeting organisational objectives, web and cloud directory applications.

JSON-RPC 2.0 Base flies with the new JSON library

JSON-RPC 2.0Two years after its initial release the JSON-RPC 2.0 Base library switches to a new package for representing and parsing JSON messages. The result is 2x to 4x times faster performance in JSON-RPC message parsing!

Back in 2009 I chose JSON Simple for its simple and relatively non-invasive model for representing and mapping JSON entities to Java primitives and objects. Performance was also relatively good against the alternatives, so JSON Simple became the natural choice for the JSON-RPC 2.0 Base classes. JSON Simple development became stale with time however, and more than 2 years passed without any new releases or fixes to reported bugs. This, I suspect, must have led to the user community becoming somewhat frustrated and a fork imminent.

A fork of JSON Simple did indeed arrive, in May this year, initiated by Uriel Chemouni and named JSON Smart. Initial tests showed considerable performance gains and after some email exchange with Uriel which resulted in the addition of strict JSON parsing, JSON Smart became the new library for JSON-RPC 2.0 Base.

So what do you gain with the new release?

JSON-RPC message parsing is sped up 2.4 times for the reusable JSONRPC2Parser and 3.8 times for the static parse methods.

Parse performance for JSON-RPC 2.0 messages

Moreover, the difference in terms of performance between the static parser and the instance parser is almost negligible, which means that for apps like servlets there is no need to pool parsers in order to speed up request processing.

Thanks for Uriel for making this great piece of software!

In the next few days Json2Ldap and the other products in the NimbusDS line will receive the new library, so expect more good news!