Wednesday, January 12, 2005

Why XML in XmlHTTPRequest?

The XmlHTTPRequest function was designed so that the client can get small pieces of information from the server without having to do a page refresh. Obviously, the developers thought that XML will be the format of choice for this data exchange.

Thank God they didn’t make XML validation necessary!

I say this because XML will not be the format of choice for this data exchange. Let’s see why.


Firstly, let’s see how the data flow will be in the case of XML over XmlHTTPRequest. A typical scenario is where the data lies in a database on the server, and the data is requested by the client. The steps in the process will be:
  1. Server reads the request, extracts the data from the database.
  2. Server prepares an XML document using this data.
  3. This XML is sent to the client.
  4. Client validates the XML for well-formed-ness.
  5. Client extracts data from the XML markup.
  6. Client calls functions/triggers events which use this data as parameters in some way or other.
For several reasons, this model is not optimum. The questions that come to the mind immediately are:
  • Is it necessary to prepare an XML document (step 2 above)? Isn’t that an extra step that is probably not necessary?
  • Client validation is necessary (step 4 above) to ensure that the XML parser can parse the XML. This puts heavy processing requirements on the client. Is this really necessary?
  • The data from the XML will then have to be extracted (step 5 above). This isn’t really a necessary step either if the data is not marked up as XML. So, then, is it really necessary to use XML?
Then there are more problems. In web applications, where XmlHTTPRequest is most likely to be used, speed is of utmost importance, and optimizing for fast response times will be imperative. In such cases, the amount of time taken for any kind of operation, especially if it happens over the network or the internet, has to be minutely scrutinized and optimized. In such cases, XML is probably not the best suited format for data exchange. XML files are heavy as far as sizes go. A simple text like “Hello World” (11 characters) can be transmitted over XmlHTTPRequest as 11 bytes of data. Marking this up as XML will increase this at least 4 fold (say, and that’s still a conservative estimate) even if whitespaces are stripped off the document, increasing download sizes four times, and reducing response by a factor of 4. Point is, XML comes with its overheads which cannot be tolerated in a web application.

So, in what format will data be transferred over XmlHTTPRequest? The simplest way is as a plain old string of data. This will be the most efficient as far as download speeds are concerned. You will download only 11 bytes where 11 bytes of download is required, not more. Besides, there won’t be any XML parsing or validation in the picture, making the application that much faster.

However, somehow, I don’t see data flowing over XmlHTTPRequest in the form of plain strings of data either. Instead, I think it’ll flow in the form of JavaScript code.

Let me use an example from the guy who deconstructed Google Suggest, and the way XmlHTTPRequest works there. According to him, the code returned by Google when you hit a keystroke looks like this.

sendRPCDone(frameElement, "fast bug", new Array("fast bug track", "fast bugs", "fast bug", "fast bugtrack"), new Array("793,000 results", "2,040,000 results", "6,000,000 results", "7,910 results"), new Array(""));

Basically, what this code does is call a function, with a bunch of parameters. The function called, sendRPCDone, is defined in client side JavaScript code. The parameters are generated depending on server side processing (probably from Google’s cached searches). Then, later in the code, this response text is run through an eval function to execute the function on the client side (eval(_xmlHttp.responseText);).

I think the eval function is really a boon here. It makes JavaScript as the language of choice over XML for XmlHTTPRequest. For who don’t know what eval does, it basically takes a piece of string, treats it as though it is JavaScript code, and executes it. The string could be dynamically generated by client-side code, or as in this case, returned from the server. Just what we need here!

To summarize, here’s why we’ll use JavaScript instead of XML when using XmlHTTPRequest.
  • Downloading XML data will take more time as compared to JavaScript. Depending on the amount of data to be transferred, this difference might be huge.
  • Use of XML will necessitate validation of the markup, so that a parser can read the document – a process that requires extra processing on the client-side. This is not necessary with JavaScript.
  • The XML data is not already in a position to call scripts. Client side code will have to handle that, based on event handlers or such. JavaScript, on the other hand will be ready to be executed on the client.
It's easy to see why XML will not be used as the format of choice over XmlHTTPRequest. A little thought later, JavaScript emerges as the winner as the format of choice for data exchange over XmlHTTPRequest.

Did someone say that JavaScript will be coming out of the closet this year?

15 comments:

Adam Michela said...

Rakesh,

I agree, mostly, that the typical use of XMLHttpRequest is to receive data to be used in a script, or a script itself.

For this purpose, XMLHttpRequest may not even be the EASIEST method, and it certainly is not so revolutionary.

My full time job is as a web developer for a large-scale intranet application in an IE6 (:o) environment.

For sometime, I have been using this function to pull dynamically generated scripts from the server without refreshing the page:

function pullScript()
{
var oHead = document.getElementsByTagName("HEAD")[0];
var oScript = document.createElement('script');
oScript.setAttribute('src','some.php');
oScript.setAttribute('type','text/javascript');
oHead.appendChild(oScript);
}

It suits our needs. It's not bulletproof, but I think it could be with minimal effort.

It should work with any W3C-DOM browser.

While I've no accurate means by which to benchmark this, I imagine it would be more responsive than XMLHttp+eval method. (eval() is JavaScripts most performance deficient function).

I'm not saying this is a replacement for XMLHttpRequest. In fact, I am also using XMLHttpRequest (More accurately: Msxml2.XMLHTTP) at work. I use it in a server-side table sort routine. When a sortable table column contains too much data too sort on the client, I query the application framework to do the job (it has the data in session already) and it returns the data as XHTML.

I'm just saying, for the popular purpose of XMLHttp that I see being discussed (and used) here and elsewhere, it is nothing extraordinary.

Rakesh Pai said...

Wow, Adam. That piece of code is poetry!

I was just thinking the other day about dynamically including JavaScript files, and XmlHTTPRequest wasn't the solution to it (or was too clumsy at best).

Your code, on the other hand, is just what I was looking for. Awesome work!

Anonymous said...

>>It should work with any W3C-DOM browser.
-not Safari. Not old Opera either. Maybe newest, forget offhand. I've used it for ages to send items to the shopping cart when ordering and works pretty well. (Btw, your code needs a timestamp added so not cached by IE. Also IE Mac needs a different routine; you'll crash it with that code.)

>>(eval() is JavaScripts most performance deficient function).
-why not just send the data string and and call the function with it? No eval needed.

Rakesh Pai said...

>> why not just send the data string and and call the function with it? No eval needed.

How would that work? Let's take the code in question in my post. The server returns the string which is basically that sendRPCDone function call. When that is received by the client as a result of the XmlHTTPRequest response, it'll be for all practical purposes just a piece of string. How do you get it to be treated as a piece of client-side code and execute it, without using eval?

Am I missing something?

Adam Michela said...

Anonymous,

Like I said... that is a rough piece of code tailored exclusively for an always-on IE6 only application at my place of work.

Old Opera is not W3C-DOM compliant, so obviously it will not work there. Nor will XmlHttpRequest (Same story with IE Mac). As for Safari... I can't say.

That snippet itself is not meant to represent the most thorough example of performing such an action. It contains no error handling and no fallback for non-compliant browsers.

I'm merely putting forth, in its most basic form, a long used alternative to XMLHttpRequest.

As for NOT using eval(), Rakesh is right... you don't have that option. Any string returned by XMLHttpRequest that you wish to execute as script must be evaluated as so.


Anyhow, we're all friends here... sharing thoughts and ideas... no need to remain anonymous :)

pat said...

Whether this is new or not doesn't really matter. Fact is, it is very underutilized. Which seems to be the case for various reasons:

1) noone seems to be able to come up with something that has decent cross-browser support.
2) the odd assumption that payloads will be XML
3) the apparent cross-domain restriction of hmlhttprequest

Can someone just put together some code that's reasonably cross platform that can do simple POSTs or GETs, ideally to any domain?

Anonymous said...

>>am i missing something

Yes, don't send the function name from the server, just the string and then send that to the function:

clientsideRPCFunction(serverResponseString);

Or you can set the callback function beforehand if sending an ID back and forth, or send the function name itself back and forth as the first argument, but that involves more coding obviously.

Rakesh Pai said...

Interesting technique, Anonymous. (I wish you'd leave your name, but really that is Blogger's fault - I understand.)

The next obvious question would be, is it better to eval a string, or better to parse a string and decide what client-side function to call? Which one is more efficient? Which one is faster? Which one would reduce memory leaks (which is increasingly becoming an issue with JS web apps)?

I'd think that for small scripts that essentially have very few callbacks, your technique would reign supreme. However for larger scripts with more callbacks, parsing the response string to determine the operation would be just as performance intensive as using eval, what with complex if-else ladders.

Of course, that's off the top. I have not tested it, nor could I come across anyone who has. Can anyone shed some light on this matter?

Anonymous said...

The idea of outputting lots of javascript directly from server side code is a bit of a false economy due to changes in javascript requiring rebuilding solutions. Combining with XSLT on the server side can give the flexibility to change without having to rebuild solutions, And also means you can output HTML directly.

lego said...

You may be get some information about XMLHttpRequest at the http://mcoder.atspace.org

Chris Purcell said...

Just want to say, most helpful article for me. I've been struggling to get decent performance for a week, bashing my XML-reading code this way and that. Switching to eval was a perfect solution for getting thousands of records into my Javascript in a tolerable timeframe. Magic.

William said...

but what if javascript is off?

Rakesh Pai said...

William,

This post assumes that JavaScript is available, which is around 96% of the typical Internet audience.

In case you are concerned about cases where users do not have JavaScript enabled, you should find out more about "Unobtrusive JavaScript" and "Graceful Degradation" techniques. It is a little harder, but not impossible, to cater to users that don't have JavaScript enabled.

wizardnet said...

Anonymous, did you find a way to use your method in a synchronous way?

Regards wizardnet

Anonymous said...

How will this handle the data which has image also in it?

ShareThis