Balisage Paper: The Apache Qpid XML Exchange

A Simple Example

The following code declares an XML exchange and uses an XQuery to specify that all messages with odd-numbered ids should be routed to a queue named "message_queue":

session.exchange_declare(exchange="xml", type="xml")
session.queue_declare(queue="message_queue")

binding = {}
binding["xquery"] = "./message/id mod 2 = 1"

session.exchange_bind(exchange="xml", queue="message_queue", 
routing_key="routing_key", args=binding)

A publisher can write XML messages to this message queue with no knowledge of the query, it needs only the name of the XML exchange so it can specify the destination:

for i in range(10):
  message = Content("<message><id>" + i + "</id></message>"))
  message["routing_key"] = "routing_key"
  session.message_transfer(destination="xml", content=message)

A consumer needs no knowledge of the publisher or the XML Exchange, it simply reads messages from its queue, as shown in the previous section.

A Weather Alert

Time to go Sailing!

Now let's explore an application that uses the XML Exchange in a publish/subscribe application using weather observations from the National Oceanic and Atmospheric Administration's National Weather Service NWS. In this application, each individual can set up a persistent queue, to which alerts are sent, and identify weather conditions that should trigger an alert. For instance, I like to sail small sailboats, and I live in Durham, NC, so I might ask for a weather alert whenever it is really good weather for sailing.

This example is composed of two programs:

In this application, a producer posts messages to an XML exchange named "weather" in the National Weather Service's standard XML format. Here is one message in that format:

<current_observation version="1.0"
xsi:noNamespaceSchemaLocation="http://www.weather.gov/data/current_obscurrent_observation.xsd">
<credit>NOAA's National Weather Service</credit>
<credit_URL>http://weather.gov/</credit_URL>
<image>
<url>http://weather.gov/images/xml_logo.gif</url>
<title>NOAA's National Weather Service</title>
<link>http://weather.gov</link>
</image>
<suggested_pickup>15 minutes after the hour</suggested_pickup>
<suggested_pickup_period>60</suggested_pickup_period>
<location>Raleigh-Durham International Airport, NC</location>
<station_id>KRDU</station_id>
<latitude>35.88</latitude>
<longitude>-78.78</longitude>
<observation_time>Last Updated on Apr 21, 2:51 pm EDT</observation_time>
<observation_time_rfc822>Mon, 21 Apr 2008 14:51:00 -0400 EDT</observation_time_rfc822>
<weather>Mostly Cloudy</weather>
<temperature_string>67 F (19 C)</temperature_string>
<temp_f>67</temp_f>
<temp_c>19</temp_c>
<relative_humidity>57</relative_humidity>
<wind_string>Variable at 7 MPH</wind_string>
<wind_dir>Variable</wind_dir>
<wind_degrees>999</wind_degrees>
<wind_mph>6.9</wind_mph>
<wind_gust_mph>NA</wind_gust_mph>
<pressure_string>29.95" (1014.0 mb)</pressure_string>
<pressure_mb>1014.0</pressure_mb>
<pressure_in>29.95</pressure_in>
<dewpoint_string>51 F (11 C)</dewpoint_string>
<dewpoint_f>51</dewpoint_f>
<dewpoint_c>11</dewpoint_c>
<heat_index_string>NA</heat_index_string>
<heat_index_f>NA</heat_index_f>
<heat_index_c>NA</heat_index_c>
<windchill_string>NA</windchill_string>
<windchill_f>NA</windchill_f>
<windchill_c>NA</windchill_c>
<visibility_mi>10.00</visibility_mi>
<icon_url_base>http://weather.gov/weather/images/fcicons/</icon_url_base>
<icon_url_name>bkn.jpg</icon_url_name>
<two_day_history_url>http://www.weather.gov/data/obhistory/KRDU.html</two_day_history_url>
<ob_url>http://www.nws.noaa.gov/data/METAR/KRDU.1.txt</ob_url>
<disclaimer_url>http://weather.gov/disclaimer.html</disclaimer_url>
<copyright_url>http://weather.gov/disclaimer.html</copyright_url>
<privacy_policy_url>http://weather.gov/notice.html</privacy_policy_url>
</current_observation>

The message consumer is an application that allows me to subscribe to this weather feed, specifying conditions for my alerts based on the data available in such a message. On my little sailboats, I especially like to sail when the local weather is at least 60 degrees Fahrenheit, the wind is between 7 and 15 miles per hour, and the temperature is at least 10 degrees above the dewpoint (so it is unlikely to rain). The application would have a GUI to allow me to specify these kinds of conditions, and would generate an XQuery for the XML binding. Here is a query that expresses these conditions:

let $obs := ./current_observation
return $obs/station_id = 'KRDU'
   and $obs/wind_mph >= 7
   and $obs/wind_mph <= 20
   and $obs/temp_f > 60
   and $obs/temp_f - $obs/dewpoint_f >= 10

Of course, this is very individual. My friend Jim has a 40 foot sailboat in Clearwater, and his sailboat just starts to get moving at speeds that would be very challenging in my little sailboat. So he might specify very different values for wind speeds. And I have other friends whose weather-related interests have nothing to do with sailing - perhaps someone would simply like an alert when it looks like it will rain. Each of us can declare an exclusive queue with a binding that expresses which messages we are interested in, as illustrated in the following code:

# Use a UUID as the name of the queue
# to guarantee a unique queue name
uuid = uuid()
session.queue_declare(queue=uuid, exclusive=True)

binding = {}
binding["xquery"] = """
  let $obs := ./current_observation
    return $obs/station_id = 'KRDU'
      and $obs/wind_mph >= 7
      and $obs/wind_mph <= 20
      and $obs/temp_f > 60
      and $obs/temp_f - $obs/dewpoint_f >= 10"""

session.exchange_bind(exchange="xml", queue="message_queue", 
routing_key="routing_key", args=binding)

Querying Message Properties

When the data used by clients is flat, and clients are interested in the same properties, a message producer can bind these properties to a message so they can be accessed without parsing the message content, which may be faster, since the message content need not be parsed. When used in this way, the XML Exchange operates much like Java JMS selectors.

In the XQuery used for binding, message properties are bound as external variables of type string. The query must declare these external variables before using them. The XML Exchange binds the message properties to the corresponding variables before the query is invoked. Here is the query from the previous example expressed using message properties instead of XML message content:

declare variable $station_id as xs:string external;
declare variable $wind_mph as xs:string external;
declare variable $temp_mph as xs:string external;
$station_id = 'KRDU'
and xs:decimal($wind_mph) >= 7
and xs:decimal($wind_mph) <= 20
and xs:decimal($temp_f) > 60
and xs:decimal($temp_f) - xs:decimal($dewpoint_f) >= 10

Properties and XML content can both be used in the same query. In the following query, the station identifier is read from the message content, but the wind speed and temperature are read from message properties:

declare variable $wind_mph as xs:string external;
declare variable $temp_mph as xs:string external;
./current_observation/station_id = 'KRDU'
and xs:decimal($wind_mph) >= 7
and xs:decimal($wind_mph) <= 20
and xs:decimal($temp_f) > 60
and xs:decimal($temp_f) - xs:decimal($dewpoint_f) >= 10

XML Messaging in the REST/SOAP/ESB Landscape

XML applications already have a bewildering number of choices for exchanging data. Many applications already use REST Fielding, SOAP Soap1 Soap2, ebXML ebXML, Enterprise Service Buses ESBWiki, Java JMS Messaging JMS, or any one of the dozens of other available choices now available. Each of these sytems has advantages and disadvantages, and a thorough comparison to these systems is beyond the scope of this paper. This section attempts to point out some of the fundamental design characteristics of AMQP and the XML Exchange to make it easier for readers to make their own comparisons.

Like REST, AMQP is simple^[5], open, platform independent, and programming language independent. However, REST is designed only for client/server request/response interaction, and because it relies on HTTP, it is a synchronous protocol. AMQP supports request/response together with other common patterns such as point-to-point, broadcast, and publish-subscribe, which allows it to define communication among messaging clients, not just communication to a server. AMQP is an asynchronous protocol, which is much more suitable for high-performance systems, though synchronous patterns are also supported in the Qpid APis.

Java JMS provides broad support for these kinds of messaging patterns, but it is a Java API, with no support for other languages, and it does not define an interoperable protocol, allowing each implementation choose how to exchange data. As a result, Java JMS does not provide interoperability across languages or across messaging systems.

SOAP and Web Services support many of the same messaging patterns as AMQP, are language and platform independent, and can also be used to achieve interoperable messaging if all parties agree on the communication protocol and on a given WS-I profile. AMQP is smaller in scope and more narrowly focused on messaging per se; it is designed to be as simple as possible while still providing robust and complete support for messaging. An AMQP server would probably be an excellent basis for implementing a SOAP server. Unlike SOAP, AMQP message headers and basic routing information are not contained in XML. This allows more efficient processing, since XML parsing is slow compared to the speed with which routing decisions are made in high-performance messaging systems, and it also cleanly separates message content from envelope information.^[6] Another difference is that XML is just one of the formats supported by AMQP, and messages can be in any desired format (but the XML Exchange was designed for XML messages). Another advantage of AMQP is that there is just one specification, and only one way to claim conformance, so interoperability does not require agreeing on specific protocols. This is simpler and more interoperable than the SOAP family of specifications WSActivity Oasis WS-I, which specify basic messaging functionality in a variety of unrelated and overlapping specifications, from different organizations, while other organizations specify profiles that can be used for interoperability.

Much of what was said in the previous paragraph also applies to ebXML. AMQP would be an excellent technology for implementing ebXML messaging, and can be used as a bridging technology between ebXML systems and other systems. But AMQP does not attempt to address anything beyond messaging.

Similarly, an Enterprise Service Bus generally refers to a more complex system build on top of an enterprise messaging system. AMQP is such an enterprise messaging system. But XML-based routing is a prominent feature in some ESB systems, and the XML Exchange provides this feature, inegrated into an AMQP server.

Summary

AMQP provides a coherent, simple architecture for high performance, reliable messaging that is interoperable across implementations and supports most common messaging paradigms. Apache Qpid is a multi-platform, multi-language implementation of AMQP. The Qpid XML Exchange provides XML-based routing in an AMQP server, using XQuery to express routing criteria based on message content and message properties. Together, they provide the simplicity of REST together with the broad support for enterprise messaging that SOAP aspires to achieve. They provide a solid foundation for mission critical XML messaging applications, and vastly simplify the task of writing XML messaging software.