Category Archives: Json2Ldap

Plain SASL authentication

Json2Ldap iconYesterday’s 2.0 release of Json2Ldap brings a lot of new good things. Some of them are hidden, representing various little stubs under the hood that will enable cool new features to be added in future (patience, you’ll find out in due time!). On the outside, the most noticeable addition is the arrival of plain SASL authentication support. This LDAP authentication method offers two key advantages over the traditional simple bind. Here I’ll explain what these are and how you could apply them.

Simpler authentication with username instead of DN

The traditional ldap.simpleBind method authenticates users with their distinguished name (DN) and password. Using DNs for web app login, however, is a bit cumbersome. Why? Because you first have to establish a search connection (bound as some service user) in order to resolve the user’s DN from the entered username or email address, typically with a filter like "|(uid=%u)(mail=%u)", and only then you can make the actual ldap.simpleBind request.

ldap.plainBind spares us these preliminary steps by taking care of the underlying user DN resolution. With it you can just pass the user’s name / email + password and be authenticated. The only snag is that the LDAP directory may require some configuration before that, to allow plain SASL binds and to know which user record attributes will serve as “username”.

Here is an example plain SASL auth included in the ldap.connect request:

{ "method"  : "ldap.connect",
  "params"  : { "host"      : "ds.wonderland.net",
                "port"      : 389,
	        "security"  : "StartTLS",
		"plainBind" : { "username" : "alice",
		                "password" : "secret" },
  "id"      : 1,
  "jsonrpc" : "2.0" }

Proxied authorisation / switching the current user

Proxied authorisation is similar to the sudo command on Unix systems. It allows you to authenticate to the directory with the credentials of one user and then perform all subsequent operations as another user. This is the other significant feature of plain SASL bind.

{ "method"  : "ldap.connect",
  "params"  : { "host"      : "ds.wonderland.net",
                "port"      : 389,
	        "security"  : "StartTLS",
		"plainBind" : { "username"       : "myWebApp",
		                "targetUsername" : "alice",
		                "password"       : "secret" },
  "id"      : 1,
  "jsonrpc" : "2.0" }

Where could proxied authorisation come to use?

In situations when a web app has to modify user data on behalf of another user. In the above example the JSON call authenticates as “myWebApp” but then requests the resulting connection to receive the identity (and privileges) of user “alice”.

Proxied authorisation can also be used to create custom external login procedures, for example based on OAuth 2.0 tokens. Here a helper app upon establishing the identity of the user through some other external mean authenticates to the directory as itself but then through the targetDN / targetUsername parameter requests the connection to take on the identity of the target user.

Note that enabling proxied authorisation for a particular service account requires the underlying LDAP directory to be explicitly configured for that. Check out the docs of your particular directory server for detailed instructions.

Which directory servers support PLAIN SASL bind (RFC 4616)?

  • OpenDJ / OpenDS
  • OpenLDAP
  • Apache DS
  • 389 Directory Server
  • UnboundID in-memory DS
  • Novell eDirectory
  • IBM Tivoli Directory Server

Directory servers without PLAIN SASL bind support:

  • MS Active Directory

Using data URIs with Json2Ldap

Json2Ldap iconPerson entries in a LDAP directory may contain photos. These are typically stored in a jpegPhoto attribute, defined in the standard inetOrgPerson schema. In this article I’ll show you how to display such JPEG images in the browser using Json2Ldap and a cool new HTML feature called data URIs.

Json2Ldap is a neat LDAP gateway software for web applications. It’s web API presents a set of JSON calls that JavaScript code running in a browser can address to access remote directory servers to perform tasks such as authenticating users, browsing their directory records, reading and updating attributes.

When Json2Ldap returns a directory entry it inspects the value of each attribute. If it is text, it passes it unmodified (save for escaping the special JSON chars). If the content is binary data, it encodes it into a Base64 string.

{ "DN" : "uid=alice,ou=people,dc=wonderland,dc=com",
  "cn"        : [ "Alice Wonderland" ],
  "mail"      : [ "alice@wonderland.net" ],
  "mobile"    : [ "+1 382 077 4180" ],
  "jpegPhoto" : [ "yugEMFjdQyRoxCP2KaZ041BUSctnUgY5TS7PBg==" ]
}

Your client JavaScript application will have no problems displaying the text attributes inside a web page. But how to render the JPEG image which was received as a Base64 string? What to do with it?

In the pre HTML 5 days you would have had to produce a physical JPEG URL from the received Base64 string, for example as by uploading it to a web server and then linking to it.

Fortunately, work-arounds such as this are no longer necessary. The solution is called Data URIs, a feature which was originally specified in 1998 and is now finally implemented by the latest crop of HTML 5 browsers.

How do data URIs work?

They provide an option for image data to be “in-lined” within the HTML page, instead only being able to link to it.

Here is a standard img element where the src attribute is a typical URL pointing to a file on the web server:

<img src="http://myapp.com/users/alice.jpg"/>

And here is an img with a data URI, where the image content is pasted directly into the src attribute. So when a data URI aware browser reads it, it will fetch the JPEG data from the in-lined Base64 string:

<img src="data:image/jpeg;base64,yugEMFjdQyRoxCP2KaZ041BUSctnUgY5TS7PBg=="/>

I don’t know why it took over 10 years to implement support for data URIs in browsers, but this is now a powerful technique for Ajax apps to render images on the web page which content was received through a XHR call.

Data URIs can naturally be applied to Ajax apps that interface to an LDAP directory through Json2Ldap. Rendering jpegPhoto attributes is then as simple as setting the src attribute of the desired image element to a data URI and then appending the Base64 content to it:

// We're using jQuery and the user directory entry 
// resides in the 'user' variable
$("#user-photo").attr("src", "data:image/jpeg;base64," + user.jpegPhoto[0]);

The online Json2Ldap demo makes use of inline images too. You can check it out at http://nimbusds.com/json2ldap-demo.html

The online Json2Ldap featuring data URIs for JPEG image rendering

Things to bear in mind:

While data URIs are supported by all major browsers today, some may impose a limit on the content size. IE 8 for example has a limit of 32 KB. So this technique may not work well for large images. For avatars and photos in user profiles, where 10 to 20KB are sufficient, it is going to be just fine.

Which LDAP servers support the Virtual List View (VLV) control?

Json2Ldap iconIf you’re making an LDAP search against a large directory you may potentially get a huge result set that you may rather wish to consume in chunks or “pages”. You may need this if you have a UI where results should be displayed, say, in portions of 10 or 20 at a time; also, if the LDAP server is configured with a max limit (typically in the order of 1000) on the number of entries a client may receive in a single search response.

To handle search results paging two LDAP extensions come to the rescue:

  • The Simple Paged Results control. It breaks the result set into pages of a specified size which the client fetches through a series of search requests. It is formally defined in RFC 2696. I have written an article in the past that lists the LDAP servers which support the Simple Paged Results control. This control is also supported by the Json2Ldap gateway / proxy as of version 1.7.
  • The Virtual List View control. It is the more featureful of the two as it allows clients to request an arbitrary “view” into the result set, by specifying a target offset, entry count, and even a “before” and “after” entry count. It is defined in an IETF Internet Draft, which for some reason still unknown to me hasn’t made it to standard status yet (as opposed to the Simple Paged Results control).

Note that the Virtual List View requires the results of the request to be sorted by a specified key on the server side using the Server Side Sort control (defined in RFC 2891). With the Simple Paged Results control sorting is optional.

The following screenshot, from the Json2Ldap online demo, shows an web app widget that uses the Simple Paged Result control.

LDAP search result chunking using the Simple Paged Result control

Here is a list of the directory servers that support the Virtual List View (VLV) control, in alphabetical order:

  • 389 Directory Server
  • IBM Tivoli Directory Server (since version 6.2)
  • Microsoft Active Directory (since Windows Server 2003)
  • Novell eDirectory (see note)
  • OpenDS / OpenDJ

Directory servers that don’t support the VLV control:

  • Apache DS
  • Oracle Internet Directory
  • OpenLDAP

The VLV control is perhaps not as widely supported by the various directory server vendors as the Simple Paged Results, but if you have an application that has to be able page up and down through the search results, it is indispensable.

Support for VLV in Json2Ldap from NimbusDS is planned for the next 1.10 release.

The NimbusDS line gets a speed boost

The Nimbus Directory Services product lineThe complete Nimbus Directory Services product line received a speed boost in June. The latest versions of Json2Ldap, JsonSSO and AuthService now feature a greatly improved JSON parser which speeds up decoding of incoming requests by impressive 380 per cent!

Thanks to Uriel Chemouni from the JSON Smart project for his top work!

What is next on the immediate NimbusDS roadmap?

So, expect more good news as we are heading into the summer. Here in London we had quite a promising start, warm and sunny!

How to use an LDAP directory for central listing and configuration of web services

Json2Ldap iconLDAP – compatible directories such as MS AD and OpenDJ are an excellent tool for bringing order and organising things. Businesses that make extensive use of web services may benefit from having all of them nicely listed and configured in a central location. A directory server can serve this purpose well, and with a gateway such as Json2Ldap, this can be done over the web too.

Basic web service listing

At a basic level, the directory may just list the available web services with their key usage details, such as their name, protocol and the URLs where web clients can access them.

The example below shows one such case where the services are listed under a directory branch named ou=Services,dc=NimbusDS,dc=com. Each entry is a webService class which supports attributes for storing a service’s name, URL, protocol and other detail. Directories allow their schema to be freely extended, so you can define your own class for your web services and the properties you wish to store.

Listing web services in an LDAP directory

The web services entries can be administered from the console or with LDAP GUI tools such as the excellent Apache Directory Studio.

Web service lookup

If you have a Json2Ldap instance installed the service listings may be queried by web clients, for example to dynamically look up the URL for a particular web service.

Here is a Json2Ldap ldap.getEntry request to obtain the URL of a TransactionCenter service:

{
  "method"  : "ldap.getEntry",
  "params"  : { "CID"        : "1f52b0c4-5d0d-4812-8688-8bf90b70eebf",
                "DN"         : "wsName=TransactionCenter,ou=Services,dc=NimbusDS,dc=com",
                "attributes" : "wsURL" },
  "id"      : "001",
  "jsonrpc" : "2.0"
}

The resulting JSON response:

{ "result" : { "DN"    : "wsName=TransactionCenter,ou=Services,dc=NimbusDS,dc=com",
               "wsURL" : [ "https:\/\/n1.cloudbase.net:8080\/tcenter\/"] },
  "id"      : "001",
  "jsonrpc" : "2.0" 
}

Centralised web service configuration

The administration of web services can be further centralised by storing their configuration variables in their own directory entries instead. In effect, this means projecting all configurations from your directory server. The web services would then only need to know the LDAP URL of the directory server and be provided with the appropriate credentials (username/password pair or X.509 certificate) to connect and authenticate to it.

If done on a consistent basis, having all configurations in a single place may greatly improve the manageability of your services and simplify the work of your IT administrators.

To accomplish this each web service must be provided with a set of credentials for accessing its configuration variables. ACLs may be judiciously used to limit read access to these attributes to the web service only and allow only the authorised administrator to edit them.

For the actual configuration variables, the LDAP protocol provides fair flexibility and allows storage of virtually arbitrary key/value pairs. As with the basic details example above, you can create your own schema for storing the configuration parameters of your particular web services.

Below is one such example for the TransactionCenter payment web service configuration. The directory entry contains attributes for configuring the back-end database connection as well as various variables pertaining to the web service itself.

Storing the web service configuration in its directory entry

In a future article I’ll share a few useful ideas how SaaS providers can utilise a central LDAP directory for managing subscriber accounts and multi-tenant app configurations.

LDAP directory in the cloud

Json2Ldap iconJson2Ldap hit the cloud this month.

I’ve always wanted to put up an online demo for the JSON web service for LDAP directory access and this is reality now. If you visit the NimbusDS website you’ll see a new Json2Ldap demo page where you can play with three Ajax directory applications:

  • Online employee directory: Presents a list of company employees where by clicking on a person’s name additional details are displayed.
  • Resolving group membership: An important organisational feature of directories is the ability to specify groups for things like departments, permissions and mailing lists. This app lists the available groups and resolves the associated members.
  • User authentication: Ajax form to authenticate a user’s UID and password against their LDAP directory account.

The Ajax apps were written in JavaScript and use a relatively small subset of Json2Ldap’s web API, namely the JSON requests ldap.connect, ldap.search, ldap.getEntry, ldap.simpleBind and ldap.close.

Ajax employee directory

As for the directory web service, I set up a Json2Ldap instance at CloudBees, together with an embedded in-memory LDAP directory that represents a typical corporate DIT consisting of a user base and several groups. You can view a screenshot of the directory tree to get an idea of its user and group structure. In a deployed state Json2Ldap with the in-memory directory take up 40 MB of memory which fit nicely into the free PaaS plan of CloudBees (256 MB).

The directory behind Json2Ldap can of course be any other LDAP v3 compatible server, such as Microsoft Active Directory, Novell eDirectory, OpenLDAP, OpenDJ, etc. So there’s complete flexibility here, also in terms of schema, as Json2Ldap is schema-agnostic.

How about Ajax responsiveness?

Measurements with the JSON-RPC Shell show that JSON calls to the Json2Ldap service at CloudBees are typically completed in about 150 ms. I don’t have a basis for comparison with other cloud vendors here, but I suppose responsiveness impacted by the relative datacentre location (e.g. EU vs USA) and the efficiency of the PaaS abstraction layers of CloudBees. Their PaaS is actually running on top of Amazon’s cloud infrastructure, so there may be quite many layers involved. When I compare the response time to a Json2Ldap service on the local intranet, the JSON calls here take about 15 ms to complete, which is an order of magnitude quicker. For the actual demo apps the 150 ms response time however is okay and doesn’t affect responsiveness in a noticeable way. You can of course try it out for yourself.

The next planned cloud demos will cover the other NimbusDS software products – JsonSSO and AuthService.

Search filter checking method added to the LDAP web service

Json2Ldap iconThe latest 1.8.1 release of the directory web service Json2Ldap adds a new JSON-RPC method for validating search filters. It can be used for example to check user input filter strings before making an actual ldap.search request.

This is a simple utility method that takes just one parameter – the filter string. Note that an LDAP connection is not required for this operation, so no CID (connection identifier) needs to be passed.

Example JSON request:

{ "method"  : "ldap.util.isValidFilter",
  "params"  : { "filter" : "&(objectClass=person)(givenName=Alice)" },
  "id"      : "0001",
  "jsonrpc" : "2.0" 
}

And here is the resulting JSON response indicating the filter was correct:

{ "result"  : true,
  "id"      : "0001",
  "jsonrpc" : "2.0" 
}

With this the “static” utility methods of Json2Ldap become five. The other served requests are:

The online Json2Ldap API docs have full details on these JSON calls.

Survey on SaaS user storage

Last week I put a survey in the SaaS group at LinkedIn. I was curious to find out what technology SaaS vendors chose to store the accounts of their users:

  • SQL database?
  • LDAP compatible directory?
  • A NoSQL store (e.g. MongoDB)?

First responses show relational databases to be the clear favourite. The cited factors – familiarity with the technology and its flexibility as a data store.

There were no mentions of LDAP. Neither were of the recently touted “emerging” NoSQL stores such as MongoDB.

I terms of LDAP, I think a major factor is that directory servers are considered very much to be “intranet” and “sysadmin” stuff and hence they have nothing to do in an application provider’s environment.

An LDAP directory can, however, be beneficial to a SaaS vendor. It can help reduce the friction of user provision from corporate directories to the service as both sides would effectively use the same language and semantics. And coupled with a robust web LDAP service from NimbusDS it would allow the SaaS user store to be scripted directly from the web apps.

Launch of Nimbus Directory Services

Nimbus Directory Services

On 1 May 2011 NimbusDS.com went live! Thanks to everyone who supported me in the early stages in getting the vision right and later on with concrete feedback as the site was evolving.

So what is Nimbus Directory Services?

It is the new dedicated home of all directory related software from my Dzhuvinov Software venture, namely Json2Ldap, originally conceived in 2009, and the more recent JsonSSO and AuthService products.

It is also about giving new life to existing LDAP – compatible directories on the web and in the cloud, in the spirit of NoSQL and NoXML. LDAP is a proven storage technology for bits of hierarchical data, both textual and binary, and also comes with built-in search and means for user authentication, which make for a great complete stand-alone data store. By coupling directories with a JSON web interface (through Json2Ldap) they can now also get on the web and be readily accessible from browser JavaScript XHR or other web services.

To first time visitors, here is a brief overview of the three pillar NimbusDS products:

Json2Ldap Json2Ldap
Turns any LDAP directory into a web friendly JSON-RPC service
JsonSSO JsonSSO
Single sign-on and shared session management for Ajax & Web 2.0 apps
JsonSSO AuthService
Authenticates LDAP users over the web using simple JSON

Where to from now?

Yesterday I published the 2011 and 2012 roadmaps for Json2Ldap and JsonSSO. At present development continues to be driven mostly by the requirements of the Transaction Company project. Initial conceptual work was done on creating interfaces to allow building of Json2Ldap and JsonSSO clusters on top of Memcached / Membase stores. The objective: to make them highly scalable and available services, able to handle many thousands of users with reliability and consistence.

I also started looking for one or more partners, to take over the task of NimbusDS marketing as well as to provide customer support and consulting where needed. The Transaction Company is top on my agenda now and I want to be able to focus on it as needed.

NimbusDS has also published a free and open source Java library, which can be used in resolving DNs from login usernames or email addresses using a DN substitution or search template. There is no dedicated on-line documentation yet, the package, however, comes with superb JavaDocs. You can get the DN Resolver from the NimbusDS download page.

Json2Ldap offers new LDAP utility functions

Json2Ldap iconJson2Ldap, the LDAP gateway for accessing directory servers from web devices and browser applications, now offers a new set of utility functions. These are related to distinct name (DN) processing and search filters. This new functionality is available in the latest Json2Ldap 1.8, which was released on 17 March.

Here I’ll give you a brief overview of what these functions do and how you may use them. Note that these are quite simple JSON-RPC requests that don’t require a connection to the back-end directory. They are “static” and you can use them without any prior LDAP connection set up.

DN functions

A set of functions related to distinct name (DN) processing.

  • ldap.util.isValidDN Allows checking of DN validity. Use to validate DNs input by users or read from configuration files.
  • ldap.util.normalizeDN Quick function for normalising a DN string. It removes all redundant whitespace and converts all characters to lower case (assuming the RDN component values are case-insensitive). Use if you intend to use the DN as a key in a hash map or if you want to store it in a normalised form.
  • ldap.util.compareDNs Compares two DNs for equality, taking account of multi-valued RDNs if any.  Use it to make sure two DNs are equal.

Here is an example JSON-RPC 2.0 request to validate a DN string:

{ "method"  : "ldap.util.isValidDN",
  "params"  : { "DN" : "cn=Alice Wonderland,ou=people,dc=example,dc=com" },
  "id"      : 1,
  "jsonrpc" : "2.0" }

And the corresponding JSON-RPC 2.0 response:

{ "result"  : true,
  "id"      : 1,
  "jsonrpc" : "2.0" }

Search filters

At present there is only one utility function related to search filter. I plan to add another one in the next Json2Ldap version, to validate search filters.

  • ldap.util.encodeFilterValue Encodes a search filter assertion value. Use it to escape special characters when you process user input into search templates like (!(uid=%u)(mail=%u)).

Example JSON-RPC 2.0 request:

{ "method"  : "ldap.util.encodeFilterValue",
  "params"  : { "value" : "a*b*c" },
  "id"      : 1,
  "jsonrpc" : "2.0" }

The corresponding JSON-RPC 2.0 response. Note how the asterisk chars (which normally constitute a wild-card) were escaped.

{ "result"  : "a\2ab\2ac",
  "id"      : 1,
  "jsonrpc" : "2.0" }

The complete web API documentation of Json2Ldap is available here.