Balisage Paper: Java Integration of XQuery - an Information Unit-Oriented Approach

http://qwwebservices.usgs.gov/Result/search?organization=USGS-MN&startDateLo=
01-01-2010&startDateHi=03-31-2010&mimeType=xml

A set of simple evaluations

evaluation "projectIds"

Desired result: a sorted list of project identifiers (<ProjectIdentifier>), delivered as a SortedSet<String> object. A string representation of the result might look like this:

860700319
860700379
NAWQA

evaluation “projectActivities”

Desired result: a nested map, where outer keys are project identifiers, inner keys are timestamps (obtained by concatenating <ActivityStartDate> and <Time>) and inner values are activity identifiers.

Desired result data type: Map<String, Map<String,String>>. A string representation might look like this:

860700319= (2 entries)
   2010-01-21#10:20:00=sun1dmnspl.01.01000102
   2010-02-09#09:30:00=sun1dmnspl.01.01000105
   
860700379= (1 entries)
   2010-01-22#12:01:00=sun1dmnspl.01.01000103
   
NAWQA= (2 entries)
   2010-01-20#09:20:00=sun1dmnspl.01.01000101
   2010-02-10#11:00:00=sun1dmnspl.01.01000104

evaluation “fractionResults”

Desired result: a nested map with outer keys specifying the sample kind (concatenation of <CharacteristicName> and <ResultSampleFractionText>), inner keys specifying the timestamp (concatenation of <ActivityStartDate> and <Time>) and inner values a list of result values (each one a concatenation of <ResultMeasureValue> and <MeasureUnitCode>) obtained for this combination of sample kind and time.

Desired result data type: Map<String, Map<String, String[]>>. A string representation might look like this:

…
Ammonia and ammonium/Dissolved=
   2010-01-20#09:20:00=
      0.49 [mg/l NH4]
      0.384 [mg/l as N]
   2010-01-21#10:20:00=
      0.283 [mg/l as N]
      0.36 [mg/l NH4]
   2010-01-22#12:01:00=
      0.08 [mg/l NH4]
      0.062 [mg/l as N]
   2010-02-09#09:30:00=
      0.307 [mg/l as N]
      0.40 [mg/l NH4]
   2010-02-10#11:00:00=
      0.55 [mg/l NH4]
      0.431 [mg/l as N]
   
Ammonia and ammonium/Total=
   2010-01-21#10:20:00=
      0.29 [mg/l as N]
      0.38 [mg/l NH4]
   2010-02-09#09:30:00=
      0.33 [mg/l as N]
      0.42 [mg/l NH4]
…

evaluation “activityResults”

Desired result: a map associating UsgsResult object arrays with string keys. The key specifies the activity identifier and the objects are custom objects representing an activity as described by one <Activity> element. It is assumed that UsgsResult objects can be loaded from <Activity> element information items.

Desired result data type: Map<String, UsgsResult[]>. A string representation of the result might look like this:

key=sun1dmnspl.01.01000101
result object:
============================================
ResultValueTypeName=Calculated
ResultStatusIdentifier=Preliminary
MeasureUnitCode=mg/l NH4
AnalysisStartDate=--
ResultSampleFractionText=Dissolved
CharacteristicName=Ammonia and ammonium
PreparationStartDate=--
USGSPCode=71846
ResultMeasureValue=0.49

...

Obtaining the results via info tray

XQJPLUS offers the possibility to let the query create an info tray, a collection of information units which may be conveniently accessed. The following code snippet demonstrates how the evaluations defined in the preceding section might be performed from a Java developer’s perspective.

// *** prepare query
// *****************
xq.registerQueryFile(queryName, queryFile);

// *** load info tray
// ******************
InfoTray tray = xq.execQuery2InfoTray(queryName, new FileInputStream(inputFile), null);

// *** read results
// *****************
SortedSet<String> projectIds =                       tray.getStringSortedSetObject("projectIds");
Map<String, Map<String, String>> projectActivities = tray.getNestedMapString2StringObject("projectActivities");
Map<String, Map<String, String[]>> fractionResults = tray.getNestedMapString2StringsObject("fractionResults");
Map<String, Object[]> activityResults =              tray.getMapString2CustomObjectsObject("activityResults");

// *** process results
// ******************* 
// ...

The various results are “information units” which the Java code may conveniently pick from the tray: selecting the appropriate getter method (depending on the unit’s data type) and specifying the result name as a parameter.

Obtaining the results via generated info shape

Using an info tray, Java code can contain two kinds of errors not detected at compile time: using a wrong access method for a given unit name, and using a unit name which does not occur at all in the tray. If this is an issue, one may work with info shapes instead of info trays. An info shape is a query-specific Java class plus interfaces, providing type-safe access exactly tailored to fit the query results. The Java code is generated from a succinct “tray schema” describing the result. A minimal version of such a schema would just state the names and data types of the units, as well as a distribution of the units over one or more read interfaces. See section “Tray schema” for a listing of the schema used to generate the Java resources used in this section. The schema is compiled into two Java interfaces and a back-end Java class. As an example, an excerpt of interface ProjectData looks like this:

public interface ProjectData {

//data access
    public SortedSet<String> getProjectIds() throws XQPException;
    public Map<String,Map<String,String>> getProjectActivities() throws XQPException;

//meta data access
    public Map<String,String> getProjectIdsMetaData() throws XQPException;
    public Map<String,String> getProjectActivitiesMetaData() throws XQPException;

// more signatures left out …
}

These generated Java components can be used for even simpler and safer Java access to the query results. This time, we let the query generate an info shape, rather than an info tray. The info shape is an object of the generated class ResultReport, from which we obtain the interfaces:

// *** create and load info shape
// ******************************
ResultReport resultReport = 
   xq.execQuery2InfoShape(queryName, new FileInputStream(inputFile), null, new ResultReport());
     
ResultData  resultData  = resultReport.getInterfaceResultData();
ProjectData projectData = resultReport.getInterfaceProjectData();

We use these interfaces to read the query results:

// *** read results
// ****************
SortedSet<String> projectIds                       = projectData.getProjectIds();
Map<String, Map<String, String>> projectActivities = projectData.getProjectActivities();
Map<String, Map<String, String[]>> fractionResults = resultData.getFractionResults();
Map<String, UsgsResult[]> activityResults          = resultData.getActivityResults();	

When using such an info shape, rather than an info tray, mismatches between unit name and the unit's data type are excluded. A further advantage is the delivery of custom objects as instances of their specific type, rather than instances of Object.

Information unit-oriented access

Whether working with an info tray or an info shape - the API client deals with information units, rather than information items. The units are named and have a granularity adapted to the client's requirements, rather than being dictated by the XQuery data model. Figure 1 summarizes the process which enables this information unit-oriented access.

Figure 1: From XDM items to information units

Schematic representation of how query execution is coupled to the generation of an info tray. A query – myquery.xq – is executed and produces a sequence of XDM items. These are passed to the assembly machine, which transforms them into a set of named information units. The units are stored for later delivery. Delivery is enabled by a spectrum of get methods, each one dedicated to a specific unit data type.

Summing up

The example demonstrated the possibility to supply Java code with the results of XQuery evaluations in a convenient and intuitive way. What was the price to pay for this convenience, how much effort had to be made in the XQuery layer? The query constructing the info tray (or the info shape, if the tray schema was compiled into Java code) comprises 111 lines of code, including comments. (See Appendix B for a full listing.) An experienced XQuery developer can write the code in an hour or two. Adding in the fact that on the Java developer's side no learning or understanding of XQuery was required, leads to the conclusion that the integration of XQuery into Java may offer an interesting perspective in general - and especially so if realized in an information unit-oriented way.

Concept

The term information unit should capture the idea of a convenient entity to use when accessing, processing or delivering information. Of course, what is convenient depends on the situation and on taste. But as a starting point one may postulate that XDM items – nodes and atomic values – certainly are information units, but not the only ones one might wish for. A node or, say, a string can be exactly what one wants to deal with. Note however the general limitation that the XDM model has only two ways to express aggregation: (a) by a tree structure found within the content of a node, and (b) by regarding a sequence of items as an XDM value. The first must be unravelled using tree navigation, and the second is a primitive collection approach. From an application programmer’s point of view, this may be insufficient.

Motivating example

Consider the situation that some processing requires as input a particular string-to-string mapping, for example mapping project identifiers to project names. Such a mapping can easily be represented by an XML structure, for example:

<projects>
   <project id=”…” name=”…”/>
   <project id=”…” name=”…”/>
</projects>

This ease of representation should not be confused with ease of usage. The Java type Map<String, String> is tailored exactly for the purpose of working with a string-to-string mapping. At least in situations where only key-based access is desired (as opposed to querying the sets of keys or values), the map may be considered simpler and more straightforward. After all, working with the XML representation one would have to be aware of three paths: one leading from the map root to the nodes representing map entries, one leading from the entry root to the key and one leading from the entry root to the value. (In the example, the paths would be “project”, “@id” and “@name”.) These paths would have to be supplied either by convention (for example using standardized element names “map” and “entry” and attribute names “key” and “value”), or by explicit meta information. So the XML representation needs meta information. Another aspect: the navigation from key to value can be achieved in different ways, e.g. via XPath or via DOM methods. So the XML representation is not unambiguous and explicit as to the appropriate usage style. Apart from that, the map access is probably more efficient. Taking these aspects together, one may conclude that XML’s wonderful ability to represent almost any kind of information does not imply that it is always the most appropriate format for using or processing that information. This is where “information units” come into play.

Before continuing, let's briefly compare the use of information units with data binding. Data binding replaces an XML structure by a one-to-one object representation. This is like a “push” model: the XML comes first and dictates the object representation, which is essentially a translation of the item tree into an object tree. Information units, on the other hand, constitute a “pull” model: what comes first is the desired representation of information; it is assembled by using XDM items, rather than echoing them.

Modelling information units

An information unit encapsulates data. It should also contain information about the data, as becomes obvious when one considers an element information item as a general prototype of information units. An element has

XQJPLUS defines the following generic model of an information unit: it has

content

XQJPLUS constrains the content to be a single Java object, or an array of Java objects.

name

The name distinguishes an information unit among siblings. The concept of siblings relates to a collection of units, which is not an inherent part of the model of an information unit. XQJPLUS introduces a distinct concept – the info tray – to model collections of information units. So the name of an information unit serves to distinguish it from other units within the same info tray. Info trays will be discussed in section “Information tray (info tray)”.

meta data

Any information unit is associated with a set of named values, modelled as a QName-to-string mapping.

type description

The type description comprises three names. The contents are guaranteed to be an instance of the physical type, which is the data type defined in the programming language. Note that this may be an interface type or a base type of the data type that is used, in which case the underlying data type is only partially revealed by the physical type. The underlying type is however fully revealed by the implementation type.

The semantic type is a name meant to identify what the content “means”. For example, while the physical type may specify the content to be a string-to-string map, the semantic type might identify the contents to be a mapping of project identifier to project name. To specify this, a name like “{http://www.example.com/ns/proj}projectIdTable” might be invented. Note that the semantic type only identifies by specifying a QName, it does not define or describe. XQJPLUS does not define the interpretation of the semantic type.

Available physical types

As the results of a query execution are delivered as information units, a key question is: what Java types are available? How is the concept of an information unit implemented in terms of Java types? If we compare XQJPLUS with a restaurant where the XQuery developer is the cook and the Java developer a potential guest, the range of available Java types is like the menu. Without a glance at that range, the value of XQJPLUS cannot be assessed.

Note: The preceding subsection introduced the notion of physical types which are mapped to Java type names. In the current subsection the Java type names are used, although the physical type is defined as a QName which is usually different from the Java type name.

As a starting point, consider the Java types which the underlying XQJ API delivers.

This spectrum is included and extended by XQJPLUS.

The extension compared to XQJ has four aspects:

support for array types

The query result which XQJ delivers is one single, unstructured sequence of items. It is not possible to express any grouping of items into sub sequences. For example, if the result consists of 10 strings, there is no way to communicate explicitly that the first six strings should be viewed as one array, followed by a single item, followed by an array of three items. In XQJPLUS, on the other hand, the result could be delivered as three units, two of which are arrays.

added categories

Map types are thought to be especially useful, as XML evaluations often yield information which can be modelled as a map. The map keys are strings, the map values can be strings, booleans, nodes or custom objects (see below). Maps can be simple or nested. Collection types are probably less important than maps, due to the general support for array types. The type java.util.Properties is currently the only type of category "miscellaneous", but the addition of other types is intended. Admitting exception objects as information units enables query authors to communicate error conditions in a unified way.

custom types

Information units can be custom type objects. Custom types are (presently) required to implement the interface LoadableFromXML, which is part of the main package of XQJPLUS. It contains a single method loadFromXML which loads an object with data received as an XML node. The construction of custom type objects will be discussed in a later section.

nested units

An information unit may be an InfoTray object, which represents a collection of named information units. (See section “Information tray (info tray)” for a discussion of info trays.) This amounts to supporting a type of unit which is a tree consisting of InfoTray objects as internal nodes and non-InfoTray objects as leaf nodes. In fact, a query result as a whole can be regarded as such a tree. Note that the tree of info trays and non-tray leaves must not be confused with an XML tree. A leaf of the latter contains an atomic value; a leaf of the former may, for example, contain an XML document, or a mapping of strings to XML documents.

Information units are retrieved from an info tray by calling a specific getter method which has the unit’s physical type as return type. See section “Read access” for more information about API access to information units.

The assembly process - basics

After deciding which physical types XQJPLUS must support, we can investigate their assembly. The assembly is wholly controlled by the XQuery code. XQuery thus provides both the information content and the control information required to transform this content into a collection of information units.

Let us look at an example. The following XQuery snippet produces two result items – a control item and a data item - which together will command the assembly of an information unit consisting of a string-to-string map:

<xqjp:part name="activityTimes" 
           type="map_string_to_string_object"
           entryPath="activity"
           keyPath="@id"
           valuePath="@time"
           sematype="usgs:activityTimes" />,

<activities>{
   for $a in //Activity/ActivityDescription return
      <activity id="{$a/ActivityIdentifier}" 
                time="{concat($a/ActivityStartDate, 'T', $a/ActivityStartTime/Time)}" />
}</activities>

The control item <xqjp:part> announces the construction of an information unit with name “activityTimes” and physical type “map_string_to_string_object”. This type name corresponds to the Java type Map<String,String>. The data to be written into the map follow in the next result item, which is a data item, rather than a control item. The assembly of the information unit (a map object) from the contents of the data item (an element node) is configured by the control item’s attributes “entryPath”, “keyPath” and “valuePath”. The values of these attributes are interpreted as XPath expressions. The entry path is applied to the data item’s root and leads to the nodes representing the map entries. The other paths are applied to these nodes, one at a time, yielding a map entry’s key and value, respectively.

Another example illustrates the mapping of a result item sequence to several information units. The query generates three information units, named “projectList”, “organizationProfiles” and “countActivities". To achieve this, it produces a stream of data items with control items interspersed.

<xqjp:part name=”projectList” 
           type=”strings”/>, 
distinct-values(//ProjectIdentifier),

<xqjp:part name=”organizationProfiles” 
           type=”nodes”/>
<orgProfile>…</orgProfile>,
<orgProfile>…</orgProfile>,
<orgProfile>…</orgProfile>,

<xqjp:part name=”countActivities” type=”integer”>,
count(//ActivityIdentifier)

The examples illustrate the basic rules of the assembly process.

Rule #1: control information is placed in top-level control items

Control information is contained by top-level control items. Each result item is either a data item or a control item. Data and control information is never mixed within result items.

Rule #2: special control items define information units

Each information unit created by the query is defined by a control item (<xqjp:part> element) which specifies all unit properties: name, meta information, type (physical, semantic, implementation types). Exceptions to this rule are units defined to be an info tray. The assembly of such complex units is discussed in a special section.

Rule #3: the source data belonging to an information unit follow the unit’s definition

The source data belonging to a unit are given by all the result items which immediately follow the unit-defining control item and come before the next control item. So the basic structure of a query result generating an info tray looks like this:

...
<xqjp:part name="x" type="..."/>,
- source data items for unit "x",

<xqjp:part name="y" type="..."/>,
- source data items for unit "y",

<xqjp:part name="z" type="..."/>,
- source data items for unit "z",
...

Two exceptions to the rule are units which are themselves info trays and unit definitions which reference control variables (see next section).

Note: For a comprehensive list of all control items defined by XQJPLUS, see Appendix C.

The assembly process - advanced

Namespace context

Unit assembly is often controlled by XPath expressions. The namespace context in which to evaluate those expressions is created by special control items. Example:

<xqjp:setNamespaceContext>
   <xqjp:binding prefix="data" uri="http://qwwebservices.usgs.gov/schemas/WQX-Outbound/2_0/"/>
   <xqjp:binding prefix="eval" uri="http://www.bits-ac.com/usgs"/>
</xqjp:setNamespaceContext>

Control variables

Since the data items of the query result are the source data for unit assembly, you may want to reuse source data in the assembly of different units. A simple example is a query which produces two string-to-string maps, where the second interchanges the keys and values used by the first – for example a “code2text” map and a “text2code” map. To achieve this without repeating the source data one may use control variables. A control variable is declared by a special control item (<xqjp:setVar>), which specifies a name and minimal type information (atomic or node, single value or sequence). During the assembly process, all data items following the declaration and preceding the next control item are bound to a variable of the declared name. Thereafter, they can be referenced by unit-defining control items (<xqjp:part>) via a special attribute (“source”). The following XQuery code illustrates this possibility by generating two maps backed by the same data item. Map “activity2time” is string-to-string, whereas the inverse map “time2activity” is string-to-string[]. Both maps are assembled from the contents of control variable “activityDates”.

(: set variable "activityDates" 
 : ============================ :)
<xqjp:setVar name="activityDates" vtype="node" />,

<activityDates>{
   for $date in distinct-values(//ActivityStartDate)
   let $activities := //ActivityStartDate[. eq $date]/../ActivityIdentifier
   return 
      <activityDate date="{$date}">{$activities}</activityDate>
}</activityDates>,

(: define information units 
 : ======================== :)
<xqjp:part name="time2activities" 
           type="map_string_to_strings_object" 
           sematype="wq:timeActivities"
           impltype="java.util.TreeMap"
           source="$activityDates"
           entryPath="activityDate"
           keyPath="@date"
           valuePath="ActivityIdentifier" />,
<xqjp:part name="activity2time" 
           type="map_string_to_string_object" 
           sematype="wq:activityTimes"
           source="$activityDates"
           entryPath="activityDate/ActivityIdentifier"
           keyPath="."
           valuePath="../@date" />

Information units that are themselves trays

As an information unit may itself be an info tray, the result item stream must somehow delimit the begin and end of such an embedded tray. To do so, use a <xqjp:complexPartBegin> and a <xqjp:complexPartEnd> control item. As an embedded tray is an information unit, the begin marker also plays the role of a unit-defining item. It must specify the unit name and may specify a semantic type and meta data, just like any other unit-defining item.

The assembly process - how XQuery constructs Java custom objects

As mentioned before, an information unit may contain custom objects, that is, objects of a type which is not a Java standard type. Here are a few physical types involving custom objects:

Presently, XQJPLUS allows only those types as custom types which implement the interface LoadableFromXML. This interface is defined in the main package of XQJPLUS and contains a single method signature:

void loadFromXML(Node source);

An underlying assumption is that the objects can be built in a two-step way, first instantiating them using a parameter-less default constructor, then calling the load method. Later versions of XQJPLUS may support other categories of custom types which can be constructed using different interfaces.

The actual custom type is a special construction attribute ( “customType”). This construction attribute is always required if the physical type involves custom objects. Example:

<xqjp:part name="placeObjects" 
           type="custom_objects"
           customType="com.bits_ac.xqjplus.appl.Place"
/>,

The query code has to provide the data source in the usual way, that is, either by subsequent data items or using the “source” attribute. The XML nodes to be used for loading the custom objects must of course provide the XML data structure expected by the particular custom type’s “loadFromXML” method.

Meta data

An information unit is associated with meta data, which are a set of named values. The names are QNames, and the values are strings. The meta data fall into two groups:

Most standard meta data are automatically assigned by XQJPLUS. Custom meta data are always assigned by the unit-defining control item.

Custom meta data

Associating a unit with custom meta data is straightforward: they are specified either as attributes or as child elements of the unit-defining control item (<xqjp:part> or <xqjp:complexPartBegin>). The name of a meta data item is the respective attribute’s or element’s name; the value is its string value. If a meta data item is specified by an attribute, the attribute name must have a non-empty namespace URI. If a meta data item is specified as a child element, the element name may or may not be in a namespace.

The following example demonstrates the use of both, attributes and child elements:

<xqjp:part name="startDates" 
           type="string_sortedset_object" 
           sematype="usgs:activityStartTimes"
           bi:cr="{current-date()}" >
    <bi:min>{min($startDates)}</bi:min>
    <bi:max>{max($startDates)}</bi:max>
    <meta:documentation>The activity start dates, as reported by the NWIS result 
    service.</meta:documentation>
</xqjp:part>,)

The definition of unit “startDates” specifies three custom meta data items (“cr”, “min” and “max”) and a standard meta data item “documentation”.

Standard meta data

Standard meta data have a name in the namespace “http://www.bits-ac.com/xqjplus/meta”. With the exception of “documentation”, which is provided by the query in the same way as custom meta data, standard meta data are automatically assigned by XQJPLUS when transforming a query result into an info tray. Currently, there are four kinds of automatically assigned standard meta data: “length” and the three type specifications “ptype”, “stype”, “itype”.

Standard meta data item “length”

It contains the number of objects contained by the unit: if the unit has an array type, it is the number of array items, otherwise it is “1” or “0” depending on whether the unit actually contains data or is empty.

Standard meta data “ptype”, “stype”, “itype”

They contain the names of physical type, semantic type and implementation type, respectively. So the type information is available as meta data.

Note. The implementation type can be specified by the unit-defining control item, using attribute “impltype”. Strictly speaking, this is not meta data information, but assembly control information. Therefore the value of “itype” may depend on the query text, although it is automatically assigned.

Concept

An info tray is a collection of information units. Viewed as a component, its only properties are

The list of information units is partially ordered, as the order of two information units is irrelevant if they have different names, but relevant if they have the same name. Meta data are a set of named string values providing information about the tray. The resource ID is a QName identifying the tray instance within a set of tray instances.

The info tray concept is independent of the mechanisms of loading the tray. XQJPLUS represents the general concept of an info tray by a class InfoTray; and it models the loading from an XQuery result by providing a subclass of InfoTray implementing such a loading method. Other subclasses of InfoTray might be developed which load the tray from, say, an XSLT or XProc result, or a set of delimited files or yet other kinds of data sources. Note that the read access methods are all harboured by the InfoTray level, as they reference only the assembled information units, never any information about the assembly process. In consequence, once the tray is loaded the actual loading mechanism is invisible from the Java perspective and may be replaced at any time.

The interface of an info tray contains

The validation of an info tray will be dealt with in section “Tray validation”.

The load process

An info tray is loaded by adding information units one by one. The intended pattern is to let subclasses of InfoTray define a data source-specific load method which assembles the units in a source-specific way and adds them to the tray by calling the base class’s add methods.

add methods

The source-specific loading is made as simple as possible by arming InfoTray with a comprehensive set of source-independent add methods. These methods receive a unit's completely assembled data content, along with meta data and unit name. For each supported physical type, there is a special add method. Some examples:

void addString(String partName, 
               String content, 
               Map<String,String> meta);

void addStrings(String partName, 
                String[] content, 
                Map<String,String> meta);

void addMapString2Node(String partName, 
                       Map<String,Node> content, 
                       Map<String,String> meta);

Note: the semantic type is communicated as a meta data item, while the physical type is implied by the choice of the method and the implementation type is determined by the actual content.

load methods

Source-dependent load methods are defined by subclasses of InfoTray. XQJPLUS provides the subclass XQueryResponse, defining a method

void loadXQuerySequence(XQSequence seq);

The method iterates over the result sequence, assembling information units as defined by the control items and adding them to the tray by calling the base class’s add methods.

Read access

The info tray offers convenient read access to its information units. Access is always to one unit at a time. The access to a unit can be grouped as follows:

The unit is usually specified by name – see below for details.

reading the unit contents

For each supported physical type of information units, there is an access method returning an instance of the physical type and accepting as input a unit name. (Use Clark notation for names in a namespace.) When there are several units with that name, the first is delivered. Examples for methods reading unit contents:

String        getString(String name);
String[]      getStrings(String name);
Node          getNode(String name);
Node[]        getNodes(String name);
Properties    getPropertiesObject(String name);
Properties[]  getPropertiesObjects(String name);
Object        getCustomObject(String name);
InfoTray      getInfoTray(String name);
Map<String,String>     getMapString2StringObject(String name);
Map<String,String[]>   getMapString2StringsObject(String name);
Map<String,String[]>[] getMapString2StringsObjects(String name);

A tray may indeed contain several units with the same name. In order to retrieve them, first obtain a cursor and then use it to iterate over those units. The code can rely on all units with a common name to have the same physical type, as a violation of this constraint would be detected during tray loading and generate an exception. In the following example code, the common physical type is a single node:

InfoTrayCursor cursor = getCursorByPartName("laboratory");
while (cursor.hasNext()) {
   Node currentUnit = getNode(cursor.next());
   // process the contents …
}

This pattern is based on the fact that for each physical type there is a second read method expecting a unit index as input, rather than a unit name. The unit index is a non-negative number identifying the unit within the tray independently of its name.

Note: When using a unit name repeatedly, each unit with that name has its own meta data, just like any other unit.

reading the unit type information

The info tray offers separate methods for reading a unit’s physical, semantic or implementation type. Only name-based access is offered, as the type information must not vary between units which have the same name.

reading the unit meta data

Unit meta data can be retrieved on bloc as a Map<String,String>, on bloc as a Map<QName,String> or itemwise, specifying the meta data item name as a QName or a Clark string. As when reading unit content, the unit is identified either by name or by passing a cursor.

reading tray meta data

Tray meta data are retrieved in a similar way as unit meta data.

Meta data

Not only the individual information units are associated with meta data, but also the info tray as a whole (and implicitly, also any info shape generated for the info tray). Standard meta data are distinguished from custom meta data.

Custom meta data

Tray-level custom meta data are specified via a dedicated control item (<xqjp:trayMeta>). The mapping of attributes and child elements to meta data items is the same as with unit-related meta data.

Standard meta data

The namespace of tray-level standard meta data is the same as of unit-level ones. Two standard meta data items can be specified by the query – “documentation” and “name”. The latter is meant to identify the specific type of a tray created by a specific query, comparable to the name of an XML document’s root element. Other standard meta data items are assigned automatically. These are:

Purpose

There are three main purposes for creating a tray schema:

Constraining the tray structure

To constrain the tray structure, the following kinds of information can be expressed:

Controlling the Java binding

If the tray is used as input for generating Java code (or other language code), the tray schema must contain further information controlling the language binding. In case of Java code, this consists of class and interface names, as well as a distribution of the unit names over the supporting interfaces.

A simple example

Here is a minimal schema from which the info shape demonstrated in section “Obtaining the results via generated info shape” was generated:

<ts:traySchema xmlns:ts="http://www.bits-ac.com/xqjplus/traySchema"
               xmlns:java="http://www.bits-ac.com/xqjplus/traySchema/java-binding">

   <!-- ============== -->
   <!-- tray structure -->
   <!-- ============== -->

   <ts:tray>
      <ts:part name="fractionResults" type="nested_map_string_to_strings_object" />
      <ts:part name="activityResults" type="map_string_to_custom_objects_object" 
               customType="com.bits_ac.xqjplus.appl.UsgsResult" />
      <ts:part name="projectIds" type="string_sortedset_object" />      
      <ts:part name="projectActivities" type="nested_map_string_to_string_object" />
   </ts:tray>

   <!-- ============ -->
   <!-- Java binding -->
   <!-- ============ -->

   <!-- implementation class -->
   <java:trayBinding className="ResultReport" 
                     package="com.bits_ac.xqjplus.appl" >

   <!-- interfaces -->
      <java:iface name="ResultData" package="com.bits_ac.xqjplus.appl">
         <java:part ref="fractionResults"/>
         <java:part ref="activityResults"/>
      </java:iface>

      <java:iface name="ProjectData" package="com.bits_ac.xqjplus.appl">
         <java:part ref="projectIds"/>
         <java:part ref="projectActivities"/>
      </java:iface>
   </java:trayBinding>

   <!-- mapping customType => className -->
   <java:customTypeBindings>
      <java:customTypeBinding customType=".*" className="$1"/>
   </java:customTypeBindings>
</ts:traySchema>

Constraining meta data

The structure part of the tray schema shown above is minimal, as it does not constrain the use of meta data. Meta data can be constrained by

The following example adds more constraints:

<ts:part name="projectIds" type="string_sortedset_object" 
      unitOccurs="+" 
      contentOccurs="?">
   <ts:meta name="bi:priorityRemark" use="required" />
   <ts:meta name="bi:projectCodeList" use="required" >
      <ts:enumeration value="internal"/>
      <ts:enumeration value="external"/>
   </ts:meta>
   <ts:meta name="bi:source" use="optional">
      <ts:pattern value="usgs-..-\d{8}.xml"/>
   </ts:meta>
</ts:part>

The “unitOccurs” attribute (which defaults to “1”) specifies that the tray may contain more than one unit with name “projectIds”. According to “contentOccurs” (also defaulting to “1”), each unit may contain zero or one data items. (Note that non-array physical types can have only “?” or “1” as value of “contentOccurs”.) Two meta data items are mandatory (“bi:priorityRemark” and “bi:projectCodeList”), for one of which value constraints are specified. The meta data item “bi:source” is optional, but when it occurs it must match the specified constraint.

By default, meta data are “open” – there may be more meta data than specified in the tray schema. To exclude this possibility, use the attribute “metaClosed" with a value of "true" on the <ts:part> element.

Tray validation

The InfoTray class offers a method validate, which validates the tray contents against a tray schema. Note that this validation ignores all schema parts relating to the language binding. If errors are detected, the validation writes one or more InfoTrayLoadingException units, so that after validation the diagnostics can be read from those units.

The main purpose of validation is to support the development process. When the query has reached maturity, it should guarantee to either deliver a tray that conforms to the agreed upon schema, or it should itself construct InfoTrayLoadingException units with appropriate diagnostics. In other words it should not leave the construction of exceptions to the validation. Following these steps for tray processing will help minimize problems:

Suggested sample code:

tray.validate();                   // remove this line when the underlying query has matured …
if (tray.hasLoadingExceptions()) { // but ALWAYS check for buffered exceptions!
   for (InfoTrayLoadingException e: tray.getLoadingExceptions()) {
   // process exception ...
} else {
   // normal processing of tray contents ...
}

Code generation

Once a tray is constrained by a tray schema, the possible use of the tray is defined exactly, and exact answers are implied to the following questions:

If, for example, the tray schema determines that the unit name “projectIds” uses a physical type “strings”, then it is clear that data retrieval for this unit name (a) is possible and (b) must use the getter variant which delivers a string array:

String[] pnames = tray.getStrings (“projectNames“);

To pass this unit name to a different getter method – say, the variant delivering a node – will result in an exception. From here it is a small step to generate a Java class which exposes the precise set of possible read methods, making their signatures error-proof by removing the name parameter as a parameter and shifting it into the method name:

String[] pnames = shape.getProjectNames();

This way, the name and physical type of the unit have become inseparable, excluding mismatches. The generated implementation of this method consists in a call of the generic tray method:

public String[] getProjectNames() throws XQPException {
   return dataSource.getStrings("projectNames");
}

Such code generation is supplied by the code generator of XQJPLUS. The code generator takes a tray schema as input and produces an info shape: a query-specific class which is backed by an InfoTray member and defines the specific access methods as described above. Along with this class, interfaces are generated according to the directions given in the tray schema.

Info shape

Whereas an info tray is a generic data container, an info shape is a specific data container, reflecting particular choices concerning the names and types of information units. Hence the name - it reflects the particular "shape" resulting from those choices. As described in the preceding section, info shapes are generated by the code generator of XQJPLUS.

To give an example, the simple tray schema shown in section “A simple example” implies the following info shape:

public class ResultReport implements LoadableFromInfoTray, ResultData, ProjectData {
    protected InfoTray dataSource;
    ...
//data access
    public Map<String,Map<String,String[]>> getFractionResults()   throws XQPException {...}
    public Map<String,UsgsResult[]>         getActivityResults()   throws XQPException {...}
    public SortedSet<String>                getProjectIds()        throws XQPException {...}
    public Map<String,Map<String,String>>   getProjectActivities() throws XQPException {...}

//meta data access
    public Map<String,String> getFractionResultsMetaData()   throws XQPException {...}
    public Map<String,String> getActivityResultsMetaData()   throws XQPException {...}
    public Map<String,String> getProjectIdsMetaData()        throws XQPException {...}
    public Map<String,String> getProjectActivitiesMetaData() throws XQPException {...}

//deliver interfaces  
    public ResultData getInterfaceResultData() {...}
    public ProjectData getInterfaceProjectData() {...}
    ...
}

The advantages for the Java developer are obvious – compiler-guarded access to the information unit with no worries about using the right names and the right physical types. Further advantage: when a unit’s physical type uses custom objects, the generated method delivers instances of the custom type, rather than of type Object:

    public Map<String,UsgsResult[]> getActivityResults() throws XQPException {...}

The tray schema controls the generation of interfaces which contain only subsets of the info shape’s read methods. This approach parcels out the results of complex XQuery processing in such a way that Java components can reduce their dependence on the tray design: if the structure of the tray is changed, for example a unit is removed or added, the impact is limited to the users of particular interfaces.