JSON-RPC 2.0 vs 1.0

JSON-RPC 2.0Users of my Json2Ldap gateway/proxy software have noticed that its client interface is based on the latest version 2.0 of the JSON-RPC protocol (actually, the 2.0 specification proposal from 2010-03-26 which has stabilised now). Version 1.0 isn’t supported at all. Why is that and what are the advantages of JSON-RPC 2.0 over 1.0?

Version 2.0 brings a number of changes to the protocol, but its single most significant new feature, for services having the API complexity of Json2Ldap and JsonSSO, are the so-called named parameters.

The original version of JSON-RPC allows for parameter passing just like the function arguments in computer languages such as C and Java – the RPC parameters are identified by their position in the enclosing JSON array, that is we have positional parameters.

Here is one JSON-RPC 1.0 request example, notice the 3 parameters:

{ "method" : "registerUser", 
  "params" : ["uid001", "Adam Brown", "abrown@example.com"], 
  "id" : 0 }

Positional parameters are great for simple method signatures that include mostly mandatory parameters and API’s that are not expected to change much in future. If you, however, have methods with numerous optional parameters, or expect to support new optional parameters in future, then using positional parameters quickly becomes impractical.

Consider for example the ldap.search call of Json2Ldap. It has a fairly complex set of parameters, with 4 mandatory and 6 optional ones:

  • Mandatory parameters: CID, baseDN, scope, filter
  • Optional parameters: attributes, derefPolicy, sizeLimit, timeLimit, typesOnly, output

A method signature using positional parameters lists the mandatory first, followed by the optional ones, preferably in an order of decreasing parameter usage. An example ldap.search request, JSON-RPC 1.0 style, with mandatory parameters only:

{ "method" : "ldap.search", 
  "params" : ["26009f7f10b38b92ca0a8744b825104f", 
              "ou=people,dc=example,dc=com", 
              "SUB",
              "(sn=Brown)"], 
   "id" : 0 }

Now the same request as above, but also specifying a value for the last optional parameter (output type = LDIF). The preceding optional parameters are passed a null to leave them at their default value.

{ "method" : "ldap.search", 
  "params" : ["26009f7f10b38b92ca0a8744b825104f", 
              "ou=people,dc=example,dc=com", 
              "SUB",
              "(sn=Brown)",
              null,
              null,
              null,
              null,
              null,
              "LDIF"], 
   "id": 0 }

Having to fill in a number of nulls for each optional parameter until we got to the optional parameter that we wanted to specify is not particularly elegant, in fact it can be error prone too. While this example may be somewhat extreme the point I wanted to make should be clear now – JSON-RPC 1.0 with its positional-parameters-only model doesn’t serve well methods with multiple optional parameters.

That’s why in summer 2009 I was delighted to find out that an improved JSON-RPC standard had been in the works, called “2.0”, bringing one really neat feature – named parameters. It allows for an alternative parameter passing method where the values are packed as members of a JSON object. Order becomes irrelevant now as each parameter is specified by name.

Yes, named parameters make JSON-RPC requests more verbose, but I think this is rather good: the requests become self-describable which helps debugging, readability and even logging.

Therefore it was a straightforward decision to settle on a JSON-RPC 2.0 interface for the Json2Ldap gateway/proxy. The 2.0 draft has significantly stabilised now and I don’t expect any changes apart from minor wording corrections by the time it becomes “official”.

Here is a repeat of the above two ldap.search calls to Json2Ldap, but using named parameters now:

{ "method" : "ldap.search", 
  "params" : {"CID":"26009f7f10b38b92ca0a8744b825104f", 
              "baseDN":"ou=people,dc=example,dc=com", 
              "scope":"SUB",
              "filter":"(sn=Brown)"}, 
   "id" : 0,
   "jsonrpc" : "2.0" }

And the request with the optional “output” parameter:

{ "method" : "ldap.search", 
  "params" : {"CID":"26009f7f10b38b92ca0a8744b825104f", 
              "baseDN":"ou=people,dc=example,dc=com", 
              "scope":"SUB",
              "filter":"(sn=Brown)"
              "output":"LDIF"}, 
   "id" : 0,
   "jsonrpc" : "2.0" }

Yes, the JSON-RPC requests look neater now with named parameters!. And as I mentioned before, readability is top too!

The JSON-RPC 2.0 improvements are not limited to adding named parameter support. The protocol was streamlined by removing some transport dependencies and specification redundancies. Also, the error reporting of common conditions (parse error, etc.) was standardised. All this makes JSON-RPC 2.0 a highly attractive choice for lightweight web communications.

PS: For my JSON-RPC 2.0 based projects I have developed my own Java implementation, called JsonRpc2 Base. It can be used both on the server and client sides and is deliberately kept as simple and minimal as possible (no frameworks, batching left out). While the software is not exactly free, its price is also minimal. The package comes with generous license terms, full source code and carefully written documentation. JsonRpc2 Base has been through a number of refinements and its current version is 1.9.1.

JsWorld 2.3.1

JsWorld iconEarlier this week I released a new minor version of the JsWorld library for localised formatting and parsing of numbers, currency and date/times in JavaScript.

So what is different in JsWorld 2.3.1?

Those of you familiar with the library know that apart from providing l10n logic, it also comes with a set of 300+ ready locale definitions which can be applied to configure the various formatting, parsing and helper classes for a particular language + country combination. These definitions I derive from the Unicode Common Locale Data Repository (CLDR) and now have been updated to its latest 1.8.1 release.

There has been a small bug since the 1.8 CLDR release, which affects the t_fmt value of the syr_SY locale (Syriac/Syria). Basically, the format string doesn’t include an AM/PM designation which makes display ambiguous and will also result in an error during parsing. I filed a ticket with Unicode and hopefully this bug will get sorted out by the next CLDR release, planned for 27 Oct. 2010.

The other change to JsWorld 2.3.1 is that I added a minified (or compressed) version of the script. There have been some requests in the past to include a minified version, so now you can have that:

  • JsWorld.js – the original script with comments and pretty formatting.
  • JsWorld.min.js – script minified with the JSMin utility by Douglas Crockford.

Otherwise, everything else is just the same with JsWorld 2.3.1. The main reason people buy the library still seems to be the comprehensive currency formatting that it offers.

Example of formatting Euros and South African Rands in the Finnish (fi_FI) locale:

<script type="text/javascript" src="JsWorld.js"></script>
<script type="text/javascript" src="locales/js/fi_FI.js"></script>

// Create locale object from fi_FI data
var locale = new jsworld.Locale(POSIX_LC.fi_FI);

// Create an Euro formatter for the fi_FI locale
var monFormatter = new jsworld.MonetaryFormatter(locale);
alert(monFormatter.format(10000));

// Create a South African Rand formatter for the fi_FI locale
monFormatter = new jsworld.MonetaryFormatter(locale, "ZAR");
alert(monFormatter.format(10000));

Running the above example produces the following formatting:

10 000,00 €
10 000,00 ZAR

Check the library manual on currency formatting if you wish to find out more.

The JsWorld website is at http://software.dzhuvinov.com/jsworld.html

Отново на Бездивен

На Гергьовден спонтанно реших да оставя всички празнични трапези и шумотевици, и заедно с моя колега от Крива спица, Наката, натоварихме колелата и още по тъмно хванахме пътя към язовира на Кърджали. Никога не съм съжалявал да зарежа кисненето и тъпченето у дома на празник като този в полза на един ден прекаран в планината. Така беше и на 3 март тази година, та и на предишния Гергьовден, когато с приятел от Габрово си направихме малък вело тур до Беленташ. Което не знаех още беше, че от трапезата все пак няма да се лишим…

По план в девет сутринта стигнахме до с. Дъждовница на северния бряг на язовира. Оттам се качихме на колелата, доста добре екипирани с резервни гуми заради множеството трънаци из района, и започнахме с яко спускане по пътечката към въжения мост. Продължихме карането си по пътечки и пътища покрай горната част на язовира, като голямата цел за деня беше връх Бездивен.

Тази част на Родопите винаги ме е очаровала. Няма нищо общо със западната, тук е съвсем друга вселена. Невероятните скални образувания, силното присъствие на камъка навсякъде и вдълбаните в него останки от древните хора, оттам идва магнетизма на този край на планината.

Към обяд, след дълго набиване на педали, влязохме и в село Русалско. Тук се отбих в малкото магазинче за да купя подарък за моите любими планинци горе под върха, единствените останали овчари в тамошната махала от десетина къщи, които миналата есен любезно ни приютиха след като замръкнахме с колелата на билото. Те отново ни посрещаха приятелски, а трепезата която бабата приготви за нас, неочакваните гости – такива вкусни и с любов приготвени ястия не бях хапвал от много време насам. Да са ни живи и здрави нашите планинци от Бездивен!

Планинците от Бездивен
Нашите планинци от Бездивен

На сбогуване се замислих. Имаме такива прекрасни хора в нашата страна, но все още допускаме да си избираме политици, които да ни разделят на лагери и да ни владеят, за техните си тесни лични интереси. Да, нашите планинци там горе не говореха добре български, а ние още по-малко турски. Но това никак не беше пречка между нас да се получи една хубава човешка връзка. Мога само да съжалявам за продължаващото откъсване на тези хора в страната ни, както и за всички ужасни предишни политики на асимилация, преименуване и изселване. Трябва да държим на всички нас тук в България, независимо какъв език говорим, в какво вярвяме или как изглеждаме. Вярвам че може да се получи. Просто ако изходим от това, че всички ние сме хора.

По въжения мостНаката на скаличките над язовираПолуостровът на крепостта ПатмосПо пътечките на НакатаГледки-и-и!!!

Завоите на р. АрдаИзкачването към БездивенНаката кара по билотоТук живеят нашите планинци

На сватбена фотосесия на НинаНик

Събота следобед присъствах на сватбена фотосесия на НинаНик в стария град на Пловдив. Моят приятел Тош от TwenKid Studio по същото снимаше кратко рекламно филмче за нейната работа.

Първомайското лънце напичаше камъните по малките улички, навсякъде беше зелено и ухаеше тежко на пролет. А настроението – то беше сватбено!

Нина е наистина сред най-добрите сватбени фотографи тук в Пловдив. Снима с усет и определено много се старае. Ако се интересувате какво се получава разгледате нейни сватбени снимки на NinaNik.com

Step out of your role

From a recent comment I made on Facebook:

People often occupy roles at work that dictate how they should “act”. But you cannot really build a trusting relationship with clients and colleagues when you pretend, when you play an artificial role. True connection can only happen when you are yourself 🙂 So it’s sometimes good to step out of your role and just be genuine 🙂

Give your LDAP server a JSON front-end!

LDAP gateway

After several months of on-and-off development, this week finally saw the release Json2Ldap, a new product for LDAP directory integration into web applications through a simple JSON + HTTP protocol.

The software has good potentials to become a success, I believe.

Json2Ldap was conceived last summer during the MyTransactionCenter project. The web application had to connect to an LDAP directory to authenticate users, retrieve employee attributes and query organisational settings such as group memberships and policies. The typical approach in such situations is to program the web server application to handle the necessary LDAP requests and then pass the processed results to the browser for display. This is the established method for working with back-end directories, but I didn’t like it, for it was out of place with the MyTransactionCenter architectural principle — to serve as a flexible aggregator of web content and web services provided by the Transaction Company ecosystem. That’s the essence of modern Ajax/Web 2.0 apps — combining various content and services to produce useful and original mashups.

The intuitive solution? I needed LDAP to “become” a web service, accessible from any JavaScript application running in a browser. This is how the idea of a JSON-to-LDAP gateway was born.

Json2Ldap accepts requests for LDAP operations which are JSON messages received via HTTP POST. JSON + XMLHttpRequest, after all, is the lingua franca in the Ajax/Web 2.0 universe. But instead of devising my own ad-hoc JSON schema, as most people do, I decided to use the JSON-RPC 2.0 protocol, which by that time had stabilised towards becoming a standard and I already had a base Java implementation for it. JSON RPC is a simple protocol for remote procedure calls (RPC), with some XML-RPC heritage, but otherwise is a lot more natural to work with from within JavaScript.

Version 1.0 of Json2Ldap provides about 35 JSON remote procedure calls, which map closely to the standard LDAP operations. It also provides several extensions.

  • Directory connection calls: ldap.connect, ldap.isConnected, ldap.close
  • Directory authentication calls: ldap.bind, ldap.anonymousBind, ldap.simpleBind
  • Directory read operation calls: ldap.getEntry, ldap.compare, ldap.search, ldap.getRootDSE
  • Directory write operation calls: ldap.add, ldap.delete, ldap.modify, ldap.modifyDN
  • Extended directory operation calls: ldap.ext.passwordModify, ldap.ext.whoAmI
  • Directory schema information calls: ldap.schema.getObjectClass, ldap.schema.getAttributeType, etc.
  • Web service information: ws.getName, ws.getVersion, ws.getTime

Json2Ldap is not just about mapping JSON RPC calls to LDAP requests. That was actually the easier bit. Considerable thought was spent to ensure that the gateway provides a well managed and securable web access to the back-end directory servers. The Json2Ldap configuration allows admins to place a number of useful access restrictions and policies, such as:

  • Define whitelists of LDAP servers that gateway clients may connect to
  • Define LDAP connection limits per client and idle timeouts
  • Require TLS/SSL for client-gateway or for gateway-directory connections
  • Require all clients to authenticate at connection time using LDAP bind
  • Refuse directory write requests
  • Predefine “canned” ldap.presetConnect and ldap.presetBind requests.

Json2Ldap was implemented as a lightweight Java web service and is packaged as a WAR file ready for deployment into any servlet container, such as Apache Tomcat. Moreover, you can deploy one gateway instance to serve all your web applications and back-end LDAP directories at the same time. This can save you tonnes of development, maintenance and admin time.

You can find out more about Json2Ldap on its product page. It comes with a rich reference describing configuration and available RPC requests in detail. In the next few weeks I plan to write a few specific usage examples and maybe a small demo application.

Contact me if you have any questions or wish to test the software at your company.

JsWorld – now with number, currency and date/time parsing

Yesterday I released a major update to the JS World library which now adds classes to parse localised number, currency and date/time strings. This feature came about thanks to a client that had purchased a JS World license and later paid for the development of reverse parsing.

The library, now under version 2.3, underwent significant refactoring in order to accommodate the parsing classes: logic that was common to the formatters and the parsers was factored out, which now made it necessary (finally) to introduce a namespace. All JS World code is now packed into a namespace object called “jsworld”. Also, a new class jsworld.Locale was introduced to take care of the validation and encapsulation of the locale properties after loading them from the external datafile.

Here is an example showing how formatting works now:

<!-- Load the JS World classes -->
<script type="text/javascript" src="JSWorld.js"></script>

<!-- Load the en_US locale properties object "POSIX_LC.en_US" -->
<script type="text/javascript" src="locales/js/en_US.js"></script>

<script type="text/javascript">
	try {
		// Create en_US locale object using the loaded locale properties
		var lc = new jsworld.Locale(POSIX_LC.en_US);

		// Create currency formatter configured for the above locale spec
		var formatter = new jsworld.MonetaryFormatter(lc);
		
		// Format a few amounts
		var amount1 = formatter.format(1500);
		var amount2 = formatter.format(-1500);
		var amount3 = formatter.format(7500000.00);
	} catch (error) {
		alert(error);
	}
</script>

And this is a JS World parse example. The steps required to initialise a parser for a particular locale are the same as for formatters.

<!-- Load the JS World classes -->
<script type="text/javascript" src="JSWorld.js"></script>

<!-- Load the de_DE locale properties object "POSIX_LC.de_DE" -->
<script type="text/javascript" src="locales/js/de_DE.js"></script>

<script type="text/javascript">

	var formattedAmount = "-1.234.567,89 €";

	// Create monetary parser for de_DE locale
	var lc = new jsworld.Locale(POSIX_LC.de_DE);
	var parser = new jsworld.MonetaryParser(lc);

	try {
		var parsedAmount = parser.parse(formattedAmount);
		document.writeln("Parsed original amount: " + parsedAmount);
	} catch (error) {
		alert(error);
	}
</script>

You can read more about parsing in the parsing manual.

Finally, JS World 2.3 also introduces a set of functions for working with ISO 8601 date/times:

JS World 2.3 is immediately available for download. The license price remains unchanged at € 495.-

Processing JSON-RPC 2.0 request parameters

Last week I updated the JSON-RPC 2.0 Base library with a set of utility classes to assist programmers with the processing of request parameters. This updated version is numbered 1.6 (2010-02-18).

The new utility classes solve three common issues that occur when processing request parameters:

  • Provide type correct retrieval of the JSON parameter values.
  • Differentiate between mandatory and optional parameters.
  • Produce an appropriate JSON-RPC 2.0 error on a missing or invalid parameter.

These points are best illustrated with a concrete example. Consider the following RPC request signature. The parameters are specified by name, as key-value pairs packed into a JSON object (called named parameters). The other way to specify the parameters is by position, by packing the values into a JSON array (called positional parameters). The parameters are of three different types (JSON string, JSON number and JSON true/false), where the first two are mandatory and last one is optional (defaults to false).

method: "makePayment"
named parameters:
	"recipient"    : string, mandatory
	"amount"       : number, mandatory
	"confirmation" : boolean, optional (defaults to false)

Let’s now create a JSON-RPC 2.0 request for the above method. Using the JSON-RPC 2.0 Base API this is as simple as

// The requested method
String method = "makePayment";

// The request ID
int id = 0;

// The named parameters
String recipient = "Penny Adams";
double amount = 175.05;
Map params = new HashMap();
params.put("recipient", recipient);
params.put("amount", amount);

// Create a new request
JSONRPC2Request request = new JSONRPC2Request(method, params, id);

To serialise the request to a JSON string just use the toString() method

String json = request.toString();

which will produce the following output (pretty formatting mine):

{
  "method"  : "makePayment",
  "params"  : { "amount" : 175.05, "recipient" : "Penny Adams" },
  "id       : 0,
  "jsonrpc" : "2.0"
}

Then, on the server side, the received JSON string is parsed back to a request object, which is then dispatched to the appropriate handler depending on the requested method name.

try {   
        request = JSONRPC2Request.parse(json);
} catch (JSONRPC2ParseException e) {
        // handle parse error ...
}

String method = request.getMethod();
// ... dispatch request to appropriate handler based on method name ...

Within the handler we can now proceed with extracting the request parameters. The utility package provides three Java classes for that:

Since the makePayment method expects named parameters (key-value map) we need to create a new NamedParamsRetriever.

// Check if we got named/JSON object params
if (request.getParamsType() != JSONRPC2ParamsType.OBJECT) {
        // Report params type error...
}
Map params = (Map)request.getParams();
NamedParamsRetriever np = new NamedParamsRetriever(params);

Now we proceed by retrieving the request parameters according to their type.

The retriever methods for mandatory parameters have the form

[return-type] getXXX(String name)

where XXX is the expected parameter type and name specifies the parameter, e.g.

boolean  getBoolean(String name)
int      getInt(String name)
float    getFloat(String name)
String   getString(String name)
...
String[] getStringArray(String name)
...

These methods will throw an JSONRPC2Error.INVALID_PARAMS if the parameter is missing or its type doesn’t match the expected. You can use this ready error object to create the appropriate JSONRPC2Response straightaway.

For optional parameters, such as “confirmation” in our example, we use methods that will return the specified default value if it isn’t set in the received request. They have the form

[return-type] getOptXXX(String name, [return-type] defaultValue)

where XXX is the expected parameter type, name specifies the parameter and the last argument the default value, e.g.

boolean  getOptBoolean(String name, boolean defaultValue)
int      getOptInt(String name, int defaultValue)
float    getOptFloat(String name, float defaultValue)
String   getOptString(String name, String defaultValue)
...
String[] getOptStringArray(String name, String[] defaultValue)
...

These methods will throw an JSONRPC2Error.INVALID_PARAMS only if the parameter’s type doesn’t match the expected.

The PositionalParamsRetriever class has similar methods, except that they accept an integer argument specifying the parameter’s position.

So, let’s proceed with the actual code now.

// Extract parameters
try {   
        // Extract mandatory "recipient" parameter
        String recipient = np.getString("recipient");

        // Extract mandatory "amount" parameter
        double amount = np.getDouble("amount");

        // Extract optional "confirmation" parameters, defaults to false
        // if undefined
        boolean confirmation = np.getOptBoolean("confirmation", false);
} catch (JSONRPC2Error err) {
         // If a mandatory parameter is missing or there is a type mismatch
         // you get a ready JSON-RPC 2.0 error object which you can use
         // to construct the appropriate response
         return new JSONRPC2Response(err, id);
}

Note that if a retrieval method fails, it will throw a ready JSONRPC2Error object which you can use to form the server’s response.

Look at the JavaDocs to see a list of all types the retriever classes support. They include all primitive types such as boolean, int and float as well as more complex types such as enumerated strings, key-value maps and arrays of primitives and strings.

Links:

JsWorld 1.4

I’ve just updated the JSWorld library to the latest world locale data derived from Unicode‘s CLDR 1.7.2 (released December 2009). The current JSWorld version is now 1.4.

You can purchase a license and download JSWorld 1.4 from here.

A newer JSWorld version is already planned and work should soon begin on it if all goes well. It will add methods to parse formatted currency amounts, numeric amounts and data/time strings.

Finally, huge thanks to the many people who contribute to the Unicode’s ongoing CLDR process of keeping an up-to-date database of the world’s various locales!

Apache Directory Studio

Recently I was evaluating various free LDAPv3 servers to store the profiles of TransactionCenter users. Apache DS was among the candidates. I did give their server a try and it seemed to fit my product’s purposes. But the greatest and most useful discovery on the Apache’s directory project website was their LDAP browser. I don’t know whether this is just a side-project or not, but I’ve been using it for 3+ months now and I can now say that this is the best LDAP client tool I have come across. And I have used a number of such software, free as well as commercial. Huge thanks to its creators!

Give it a try: Apache Directory Studio.

The client packs a lot of functionality so the GUI may appear a bit cluttered at first. But once I got used to it I trashed all other LDAP clients I had on my computer.

Apache Directory Studio
Apache Directory Studio