Retrieving documents via XMPP

Summary: this page explains how documents are loaded in Gradient using the IQ stanzas in XMPP, and includes notes on using HTTP for large documents.

Documents are loaded via XMPP in a fairly simple manner. The client sends a GET or SET IQ to the 'server' JID. This IQ contains an extension that specifies a path and the parameters (if any) of the request.

The server JID responds with either an ERROR IQ, which denotes an error either in the request or an internal server error, or with a RESULT IQ that either contains a document, or redirects the client to an alternate location from which to load the document.

Constructing a request

Consider this URL:

gradient://gradient-examples@jabber.org/svg/index.svg?param1=value1&param2=value2

If the path and parameter sections are missing then the path is assumed to be "/". No parameters are required.

This URL is used to construct a document request IQ. The corresponding parts of the URL and the IQ request have been highlighted. The thing looks a bit like an "April fools" version of HTTP over XML, but that's life:

<iq id="LWFOs-7" to="gradient-examples@jabber.org/gradient" type="get">
  <gradient:request xmlns:gradient="http://ex-337.net/xmlns/xmpp/gradient" gradient:path="/svg/index.svg">
    <p name="param1" value="value1" />
    <p name="param2" value="value2" />
  </gradient:request>
</iq>

Note that the path from the URL is in an attribute of the Gradient extension, and that the resource part of the JID is always "gradient". This is because of a slight semantic discrepancy between a Gradient URL and an XMPP JID.

What's slightly new is that this is also acceptable:

<iq id="LWFOs-7" to="gradient-examples@jabber.org/gradient" type="get">
  <gradient:request xmlns:gradient="http://ex-337.net/xmlns/xmpp/gradient" gradient:path="/svg/index.svg">
    <p name="param3">
      arbitrary text here &lt;escaped&gt; of course
      <element attributes="allowed">
        bla bla bla
      </element>
    </p>
  </gradient:request>
</iq>

That's right ladies and gentlemen, Ian thought it would be a good idea to allow parameters to be elements. One issue with this is that it's a bit difficult to serialise XML into a normal URL parameter, so requests with XML parameters can't easily be displayed as a URL. The reason this is here is do to with how documents loaded and modified over using Gradient document requests & directives will (eventually) be bookmarkable.

Once the server JID has received the document request as formatted above, one of two things may happen:

  1. The server JID may respond with an error, like so:

    <iq id="LWFOs-7" to="gradient-examples@jabber.org/gradient" type="error">
      <error code="404">Not found</error>
    </iq>

    This is a standard XMPP error.

  2. The server may return a document.

Response structure

Continuing with the current example, assume that there's nothing wrong with the server, which has returned a document. For the purposes of this document assume we're already logged into an XMPP network using the JID "me@jabber.org/squiggle".

There are two types of response:

In an inline document response, the requested document is included as part of the response IQ. In a redirected response, the client is given another URL from which to load the document.

Inline document loading

The hypothetical response received over the wire via the XMPP server (in this case jabber.org), and abbreviated for clarity, is:

<iq from='gradient-examples@jabber.org/gradient' id='LWFOs-7' to='me@jabber.org/squiggle' type='result'>
  <response xmlns='http://ex-337.net/xmlns/xmpp/gradient'
   thread='a71effbfff6ff587c4e572d944f7cd525280fc3d23be4bb1'>
    <svg height='481' width='500' xmlns='http://www.w3.org/2000/svg'>
      ...
      (approx. 11K of markup)
      ...
    </svg>
  </response>
</iq>

Features worth noting here:

There are several important issues to bear in mind when constructing SVG for transport using Gradient over XMPP. The most important one concerns hyperlinks.

As with normal hypertext documents, hyperlinks are relative to the current path. For example, this link:

<a xlink:href="../design/flowchart.svg">link</a>

Within this document:

gradient://gradient-examples@jabber.org/svg/index.svg

Resolves as follows:

gradient://gradient-examples@jabber.org/design/flowchart.svg

The first issue is when loading images or other non-XML data. At present, Gradient does not allow or understand the transfer of binary data over the XMPP network. If Gradient did understand binary/XMPP, it would be encoded as Base64. Base64 blocks embedded in XML are Bad, Wrong and Unfunny.

However, images may be linked to, or loaded, from non-Gradient URLs such as HTTP. HTTP references must be absolute in all documents loaded over Gradient, even when loaded via a redirected response.

Redirected document loading

Some SVG documents can be big. The map of Spain taken from the Batik project and used in the example site is just under 260 Kb. This may not seem like much, but bear in mind that XMPP was created with the intention of transporting instant messages, the average size of which is under 2KB. Compared to this, a map of Spain is much bigger.

In addition to this, constantly transferring documents of this size would quickly affect the "karma" metric used to rate-limit XMPP clients on public servers such as jabber.org, which would not be good.

Therefore, there is a second way of loading documents into a Gradient client, which is a lot less bandwidth-intensive for the XMPP server. As shown below:

<iq from='gradient-examples@jabber.org/gradient' id='LWFOs-7' to='me@jabber.org/squiggle' type='result'>
  <response xmlns='http://ex-337.net/xmlns/xmpp/gradient'
   thread='a71effbfff6ff587c4e572d944f7cd525280fc3d23be4bb1'>
   src='http://ex-337.net/gradient/svg/map.svg' />
  </response>
</iq>

Highlighted above is the "src" attribute, which is an HTTP URL. If a response such as the above is received, then the document is loaded via normal HTTP.

Important: links within documents loaded from a redirected response are still relative to the URL used to load the document in the first place.

Once the document is loaded, the displaying canvas will register itself with the connection PacketRouter, and the document may be updated by the server, see "Updating documents via XMPP".

© 2006. Some rights reserved. Author: Ian Sollars.