Introduction
Type introspection gives a program the ability to determine the type of data item at runtime and manipulate the type as a data item in its own right. It allows the creation of more generic and flexible data-processing frameworks, and allows operations to be constructed in a more data-driven way. Type introspection also enables data exploration and discovery. This paper explores one way of providing type introspection capabilities to XQuery, looks at some design and implementation issues, and shows the application of type introspection in various ways.
It may seem odd to talk about type introspection in the context of XQuery, given that it is not an object-oriented language, and meta-classes and type introspection are object-oriented concepts. However, XQuery does have the concept of a type hierarchy with subtypes and supertypes. Instance variables can be statically identified as being of one type, while being dynamically instances of some subtype. Instead of classic objects and base types, XQuery has XML nodes and base (atomic) types. Adding structural type introspection to XQuery brings powerful meta-programming patterns to XML.
Many programming languages offer some kind of reflection or introspection. These terms are frequently used somewhat interchangeably. Gabriel et al Gabriel91 define reflection as "the ability of a program to manipulate as data something representing the state of the program during its own execution", and divide it into two parts: introspection and effectuation. Introspection is the observational aspect of reflection; effectuation is the active aspect, the making of changes to the program state. Singh Singh04 usefully distinguishes between type identification, structural introspection, and behavioural introspection. Type identification allows a program to determine whether an object is an instance of a particular class (or type). Structural introspection provides the programmatic means to examine attributes of the class, such as its base class and members. Behavioural introspection allows for the examination of the runtime operations of a program. In this paper, we are concerned with type introspection, and so we will not address behavioural introspection further.
Some form of type identification is common
to many programming languages. For example, the Java or PHP
instanceof, the C++ dynamic_cast, the Perl
isa all provide for type identification, where
the reference type is named statically and must be known at compile time.
XQuery XQuery30 provides for this form of type
identification too, through the instance of keyword pair.
Figure 1: Type Identification Examples
/* C++ example */
QName* name = dynamic_cast<QName*>(&value);
if (name != 0) {
cout << name->namespaceURI();
}
/* Java example */
if (value instanceof QName) {
QName qn = (QName)value;
System.out.println(qn.namespaceURI());
}
(: XQuery example :)
if ($value instance of xs:QName) {
fn:namespace-uri-from-QName($value)
}
Examples of type identification in several languages.
XQuery allows for named types to be applied in several ways:
through the use of the instance of, cast as,
treat as, and (in XQuery 3.0) validate as
constructs, and through the use of sequence type matching in
typeswitch and argument or variable binding constructs. A program
can use the constructs to ensure that data is properly typed in accordance with
the assumptions of processing and dispatch to the appropriate processing for
the type.
Consider, for example, writing some code to analyze dates in
documents of various kinds, where the documents may use different kinds of
dates. Suppose further the documents come in a variety of forms and we do not
necessarily have schemas for every document, although we do have schemas
defining each date format. If a document has a schema, then a
typeswitch or instance of test could be used to
determine which analysis function to apply, and type sequence matching could be
used to ensure that the analysis code is applied properly to ensure
correctness. In other cases castable as and cast as
can be used to perform this job.
Figure 2 shows a fragment of XQuery dispatching to the
appropriate analysis code along these lines.
Figure 2: XQuery Type-Based Dispatch
import module iso="http://example.com/dates/iso";
import module vat="http://example.com/dates/vatican";
...
declare function my:process-date($date)
{
if ($date castable as iso:date)
then iso:process-date($date cast as iso:date)
else if ($date castable as vat:date)
then vat:process-date($date cast as vat:date)
(: etc. one for each format :)
...
};
XQuery fragment for dispatching based on hard-coded type tests and functions.
In all the XQuery constructs, the type must be a named type, and its properties cannot be inspected, compared, or reapplied. In particular, any annotations or facets associated with the type are not accessible to the program directly. One cannot ask for the type of one item and use it in any way to construct a new instance of that type. The code must explicitly reference the names of any types it needs.
Many languages also provide for structural introspection as well as
type identification. Java's Class class and the reflection API
Java and Perl 6's Perl6 class traits
and methods allow for structural reflection.
Use of a meta-class in this way is the common pattern
for providing structural introspection. It generally also allows for more
dynamic type introspection, as well as dynamic instance construction.
XQuery does not provide for this kind of type introspection. The rest of this
paper explores adding structural type introspection to XQuery.
Figure 3: Structural Introspection Examples
Class myClass = value.class;
Class paramTypes[] new Class[1];
paramTypes[0] = String.TYPE;
Constructor ct = myClass.getConstructor(paramTypes);
Object args[] = new Object[1];
args[0] = new String("Hello World");
Object newObject ct.newInstance(args);
Example of Java structural introspection. We get the class of a particular value, find a constructor that takes a string argument, and construct a new instance of the class.
XQuery 3.0 introduces a couple of new features that interact well with type introspection: function items and the simple mapping operator (represented with the ! character). Function items allow functions to be treated as values, assigned to variables, and passed as arguments to functions. As we will see below, in some cases we will treat types as function items for some purposes. Beyond that, however, putting function names in schema annotations makes it possible to write generic data-driven processing frameworks, by using the introspection API to obtain the name of the function to perform some particular operation, and the function lookup and function application features to execute that operation. The simple mapping operator allows for mapping of non-node values in streamlined path-like expressions. Many of the examples that follow use one or the other of these constructs. Expressions involving the simple mapping operator could be rewritten in other ways, but use of function items cannot be readily replaced in an XQuery 1.0 setting. I will point out alternatives where they exist.
Types as Values
Where should XQuery meta-classes fit into the existing classes in the type hierarchy? What do the classes look like? XQuery has a complex type system, integrating simple and complex types from XML Schema XSD11.2, node kinds from the XQuery and XPath data model DM30, as well as special item types such as function items.
Figure 4: XQuery 3.0 Type Hierarchy
The diagram shows the main branches in the XQuery 3.0 type hierarchy. Diamonds represent abstract types. See the XPath data model DM30 for more details.
XML Schema XSD11.1 defines a complex set of schema component types. Each schema component has its own collection of properties, some of which are components themselves. For example, among the properties on a simple type definition are its name, a base type definition, a primitive type definition, an indicator of constraints on deriving new subtypes, whether it is a list or union type, and a variety of facets indicating additional constraints such as the maximum permitted value.
For our purposes, adding richer type introspection to XQuery amounts to creating values that represent schema components and providing mechanisms for accessing those components and their their properties.
Values representing types have some similarities with nodes and function items. Like nodes, schema components are complex objects with many properties, such as names. However, most of the accessors for nodes do not apply to schema components and most operations on nodes do not apply to schema components. In XQuery, every simple type also creates an implicit constructor function. In this sense, simple types act as function items. We could perhaps make type values a kind of function item. However, type values are different from function items in every other way, and other schema components are not like functions at all. Values representing key schema components will be placed in the XQuery type hierarchy as a new kind of item.
There are two kinds of type definition component: simple type definitions and complex type definitions. Corresponding to each of these components is a component value, one for simple type definitions and one for complex type definitions. Since the distinction between a simple type and a complex type matters in some cases (values can only be cast to simple types, for example), but not in others (when we don't know whether a particular element has a simple type definition or a complex type definition, but we want to know its name, regardless), we introduce a superclass "type definition" that encompasses both simple type definitions and complex type definitions.
Figure 5: Type Values
Type values as component values, a new kind of item()
Integrating these new value types into XQuery involves:
-
Defining sequence type tests
-
Defining the means to obtain type values from other values
-
Determining how existing functions and operators apply to type values
-
Defining useful functions and operators over type values
-
Defining accessors on type values
Let's look at each of these steps in turn.
Type Tests for Type Values
For each kind of component value there is a sequence type test that matches values of that kind. Since types might have names, sequence type tests for type values can have a QName and will match if either the type has that name or it has a base type that matches.
Figure 6: Sequence Type Tests: Types
|
|
Matches any component value. |
|
|
Matches any type component value. |
|
|
Matches any type component whose name matches the given QName, or which is a subtype of a component with that type. |
|
|
Matches any simple type component value. |
|
|
Matches any simple type component whose name matches the given QName, or which is a subtype of a component with that type. |
|
|
Matches any complex type component value. |
|
|
Matches any complex type component whose name matches the given QName, or which is a subtype of a component with that type. |
Obtaining Type Values
Linkage functions get component values from various kinds of
items. They are defined over all kinds of items, returning the empty sequence
if necessary. These functions all take any item as a parameter and have a
zero-argument form that applies to the context item. This is a small usability
feature, but makes such expressions as
/root/child/sc:simple-type() possible.
In this paper, the schema component linkage and accessor functions
are all defined as being in the namespace bound to the prefix sc
which the examples assume to be predefined. The specific namespace chosen is
not important. The experimental implementation used
http://marklogic.com/xdmp/schema-components.
Figure 7: Accessors Linking from Items to Component Values
|
|
Get the type of the item as a value. |
|
|
Get the simple type of the item as a value. |
|
|
Get the complex type of the item as a value. |
XQuery introduces an interesting wrinkle that is not found in most
other programming language type systems: some data may be untyped. The special
types xs:untyped and xs:untypedAtomic represent such
untyped complex and simple content.
For atomic values that are instances of a particular simple type,
the sc:simple-type linkage function returns that simple type as a
component value. If the atomic type is untyped, then a simple type definition value
for xs:untypedAtomic is returned. The accessor
sc:simple-type operates on attribute nodes in a similar fashion.
The operation on element nodes is similar, except that if
the element is an instance of a complex type, sc:simple-type
returns the empty sequence. For other kinds of items, the empty sequence is returned.
The linkage function sc:complex-type works analogously
like sc:simple-type except atomic values and attributes can never
be instances of a complex type, so sc:complex-type is only useful
for element nodes. If an element node is an instance of a specific
complex type, the value will be that complex type as a component value. If the
element node is untyped, xs:untyped will be returned as a complex type
component value. When an element is an instance of a simple type,
sc:complex-type returns xs:anyType as a complex type component
value. For other items, the empty sequence is returned.
The sc:type accessor combines certain aspects of the
sc:simple-type and sc:complex-type accessors. It
returns a type definition of the appropriate kind. For untyped element nodes,
xs:anyType is returned as a type definition value.
Functions and Operators on Type Values
Obtaining types as values is well and good, but what can we do with
them other than perform sequence type tests on them? At a minimum, equality
operators should work on these values: equals, =, and
!=. It is also useful to be able to test whether one type is a
subtype of another.
Figure 8: Operators
|
|
Test component equality. |
|
|
Returns true if the first type is a subtype of the second. |
declare function my:spaceship-operator ($t1 as type(), $t2 as type())
{
if ($t1 subtype of $t2) then -1
else if ($t1 = $t2) then 0
else if ($t1 subtype of $t2) then 1
else xs:double("NaN")
};
XQuery F&O30 defines a large number of
functions. However, only a few
apply to all items. The sequence functions, such as fn:subsequence
can work on sequences of component values with no problem.
The general functions fn:string,
fn:boolean and fn:not don't really apply and will
raise errors if applied to component values.
If we have equality on type component values, then not only can we
define fn:deep-equal over them, we can define
fn:deep-equal with them. Figure 9 shows a
version of deep-equal that is stronger than that defined by
XQuery: it requires type equality as well as structural equality.
Figure 9: Implementing deep-equal with Type Values
declare function my:deep-equal ($p1 as item()*, $p2 as item()*, $coll as xs:string)
as xs:boolean
{
let $c1 := count($p1)
let $c2 := count($p2)
return
if ($c1 != $c2) then false()
else if ($c1 != 1) then
every $i1 at $i in $p1 satisfies my:deep-equal($i1, $p2[$i], $coll)
else if (exists(sc:simple-type($p1)) and exists(sc:simple-type($p2))) then
sc:simple-type($p1) eq sc:simple-type($p2) and
if (($p1 instance of xs:string) and ($p2 instance of xs:string))
then compare($p1, $p2, $coll) eq 0
else
try {
let $v := $p1 eq $p2
return ($p1 eq $p1) or (($p1 ne $p1) and ($p2 ne $p2)) (: NaN :)
} catch * {
false()
}
else typeswitch ($p1)
case document-node() return
(typeswitch ($p2)
case document-node() return true() default return false()) and
my:deep-equal($p1/(*|text()), $p2/(*|text()), $coll)
case element() return
(typeswitch ($p2)
case element() return true() default return false()) and
node-name($p1) eq node-name($p2) and
exists(sc:type($p1)) and exists(sc:type($p2)) and
sc:type($p1) eq sc:type($p2) and
my:deep-equal( for $a1 in $p1/@* order by name($a1),
for $a2 in $p1/@* order by name($a2), $coll ) and
if (exists(sc:simple-type($p1)))
then my:deep-equal(data($p1), data($p2), $coll)
else my:deep-equal($p1/(*|text()), $p2/(*|text()), $coll)
case attribute() return
(typeswitch ($p2)
case attribute() return true() default return false()) and
node-name($p1) eq node-name($p2) and
my:deep-equal(data($p1),data($p2),$coll)
case processing-instruction() return
(typeswitch ($p2)
case processing-instruction() return true() default return false()) and
node-name($p1) eq node-name($p2) and
compare(string($p1), string($p2), $coll)
case namespace-node() return
(typeswitch ($p2)
case namespace-node() return true() default return false()) and
my:deep-equal(node-name($p1), node-name($p2), $coll) and
compare( string($p1), string($p2), $coll ) eq 0
case text() return
(typeswitch ($p2) case text() return true() default return false()) and
compare( string($p1), string($p2), $coll ) eq 0
case comment() return
(typeswitch ($p2) case comment() return true() default return false()) and
compare(string($p1), string($p2), $coll)
case function() return error("err:FOTY0015")
case component() return $p1 eq $p2
default return false()
};
Once types are available as values, they can be returned in place of type names to perform introspection on functions, for example.
Figure 10: Miscellaneous Introspection Functions
|
|
Return a type value for type with the given name, if any. Raises an undefined type error otherwise. |
|
|
Return a type value for the declared return type of the given function. |
|
|
Return a type value for the declared type of indicated parameter of the given function. Raises an error if there is no such parameter. |
Type Values as Functions
In XQuery, there is a constructor function for each in-scope (named)
atomic type derived from a primitive type, whether the type is user-defined or
built-in. This excludes list and union types. Given that we have a schema
component value in our hands, there is no particular reason to require the type
to have a name, or even be atomic. Indeed, if one generalizes the concept of
the "constructor" function to mean obtaining a typed value of the given type
or raising an error if that is not possible, there is no particular reason
not to define functions for complex types as as well, where the "constructor"
function for a complex type validates an item against that type. The
only types where this doesn't make sense is the "untyped" types
(xs:untyped and xs:untypedAtomic), which can be
applied as no-ops. A function can be defined to perform this typed-value
fabrication operation:
Figure 11: Types as functions
|
|
Apply the type to the item, to obtain a typed value or an error. |
Figure 12: Example of applying a Type
declare function local:check($old as item(), $new as item())
{
let $f := $old/sc:type()
return sc:type-apply($f,$new)
};
local:check(doc("config.xml")/config/language, "en"),
local:check(doc("config.xml")/config/encoding, "utf-8")
With the XQuery 3.0 data model, we can go further, as function items are defined as values. Then we can just treat types as function items and apply them directly, as shown in Figure 13. Note that this might be an anonymous type just as well as a named type, unlike the type identification features that are part of standard XQuery.
Figure 13: Example of applying a type as an XQuery 3.0 function item
declare function local:check($old as item(), $new as item())
{
let $f := $old/sc:type()
return $f($new)
};
local:check(doc("config.xml")/config/language, "en"),
local:check(doc("config.xml")/config/encoding, "utf-8")
Combined with some of the introspection functions, we can get even
more powerful and flexible effects. The function run-plugin can
guarantee that a plugin function executed for some particular piece of the
framework has the expected type, without having to know a priori what that type
is.
Figure 14: Using Type Introspection in Plugin Framework
declare my:run-plugin($config as element(), $plugin as function(*), $args as item()*)
{
let $expected := sc:type-named(data($config))
if (sc:return-type($plugin) = $expected)
$plugin($args)
else
$expected($plugin($args))
};
Component Accessors
For true meta-programming, being able to access some of the details of the type components is important. Schema components are complex and have many interconnections and properties. Attempting to provide a clean API to access all these properties is beyond the scope of this paper. We will content ourselves with defining a few basic accessors.
Component accessors work analogously to accessors for nodes in the data model: they apply to all component types, returning the empty sequence where necessary. As with the linkage functions, these functions have a zero-argument form that applies to the context item. This will make chaining them together a little more usable.
Figure 15: Basic Component Accessors
|
|
Return the name of the component, if any. |
|
|
Return annotations on the component as a sequence
of |
|
|
Return the base type of the component, if any. |
Getting at schema annotations from instances has many applications. For example, schema annotations can be used to define business rules, widget bindings for generic UI generation, associating help text with input components, defining special indexing for certain content, defining XML mapping rules, and so on.
Figure 16: Some Schema Annotations
<xs:simpleType name="address">
<xs:annotation>
<xs:documentation>
A numeric internet address.
</xs:documentation>
</xs:annotation>
<xs:restriction base="xs:token">
<xs:pattern value="[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+"/>
</xs:restriction>
</xs:simpleType>
<xs:element name="address" type="address">
<xs:annotation>
<xs:documentation>
The server socket bind numeric internet address.
</xs:documentation>
<xs:appinfo>
<admin:default>"0.0.0.0"</admin:default>
<admin:cold-restart/>
<admin:help xmlns="http://www.w3.org/1999/xhtml">
<span class="help-text"><span style="color:#800000">address</span>
specifies the IP address for the App Server.</span>
</admin:help>
</xs:appinfo>
</xs:annotation>
</xs:element>
Schema annotations from a real product.
One key difference between XML types and classes in an object-oriented programming language is that classes have methods as well as data. Combining function items with annotation accessors can bring methods to XML types as well, in a way.
Figure 17: Function Items Named in Annotations as Methods
The schema:
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:record="http://example.com/meta/records"
xmlns:base="http://example.com/meta"
targetNamespace="http://example.com/meta/records"
elementFormDefault="qualified">
<xs:import namespace="http://example.com/meta"
schemaLocation="meta.xsd"/>
<xs:simpleType name="studio">
<xs:annotation>
<xs:appinfo>
<base:input-handler>record:normalize-studio</base:input-handler>
<base:formatter>record:format-studio</base:formatter>
</xs:appinfo>
</xs:annotation>
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:complexType name="record-metadata">
<xs:complexContent>
<xs:extension base="base:metadata">
<xs:sequence>
<xs:element ref="studio" type="studio"/>
</xs:sequence>
<xs:extension>
</xs:complexContent>
</xs:complexType>
Formatter function using metadata "methods":
declare function base:format-metadata ($metadata as element())
{
for $meta in $metadata/*
let $methods := $meta/sc:element-decl()!sc:annotations()/xs:appinfo
let $formatter := function-lookup($methods/base:formatter, 1)
return $formatter($meta)
};
Other Schema Component Values
Facets
Accessing facet information is important for working with simple
types. Figure 18 summarizes the additional type tests
and accessors for working with facets. The sc:facets accessor
returns sequences of facet component values when applied to simple type
components; it returns the empty sequence otherwise. The accessors already
defined apply to facet component values as well. The sc:name
accessor returns the name of the element used to define the facet
(e.g. xs:enumeration for the enumeration facet), and the standard accessor
fn:data returns the value of the facet.
Figure 18: Additions for Facet Values
|
|
Type test matching any facet value. |
|
|
Type test matching any facet value whose name matches the given QName. For example, a pattern facet matches 'facet(xs:pattern)'. |
|
|
Get all the facets of the item, if any. Only simple types have facets. |
Let's return to the date example from the introduction (Figure 2). With access to types and their facets, as well as function lookup and function application, we can create a generic framework for processing dates without having to hardcode any type or function names. A fragment of this framework is shown in Figure 19.
Figure 19: Generic XQuery Type-Based Dispatch
...
declare function my:process-date($date)
{
for $date-type in fn:collection("templates")/date/sc:simple-type()
let $date-pattern := sc:facets($date-type)[. instance of facet(xs:pattern)]
let $process-func :=
fn:function-lookup(fn:data(sc:annotations($date-type)//my:date-func), 1)
where fn:exists($date-pattern) and
fn:exists($process-func) and
fn:matches($date, fn:data($date-pattern))
return $process-func($date-type($date))
};
XQuery fragment using schema annotations and function lookup to create a generic date-processing framework.
Element and Attribute Declarations
As the example in Figure 16 shows,
it is frequently more natural to obtain the annotations from element and
attribute declarations rather than from the types associated with those
declarations, Annotations often have more to do with the use of a type in a
particular context than with the type itself. Adding accessors to obtain element and
attribute
declarations from element and attribute nodes, and defining the component
accessors over these new component types allows for these use cases.
Figure 20 summarizes the additional type tests and
accessors for working with element and attribute declarations. The
sc:element-decl and sc:attribute-decl
accessors return component values when applied to element and attribute nodes,
respectively; otherwise they return empty sequence. In addition, the accessors
already defined apply to these new declaration component values as well. The
sc:type, sc:complex-type, and
sc:simple-type accessors return the type associated with the declaration.
Figure 20: Additions for Element and Attribute Declaration Values
|
|
Type test matching any element declaration value. |
|
|
Type test matching any element declaration value whose name matches the given QName, or which is in the substitution group with an element with the given QName as its head. |
|
|
Type test matching any attribute declaration value. |
|
|
Type test matching any attribute declaration value whose name matches the given QName. |
|
|
Get the element declaration of the item, if any. |
|
|
Get the attribute declaration of the item, if any. |
Fitting element declaration values into the type hierarchy leads us to go still further: element declarations are but one kind of term in a content model, along with wildcards and model groups. Given that we already have element declaration values, it makes sense to expose the content model of a type in general. Given that we already have attribute declaration values, it makes sense to expose the attributes declared on a complex type.
The XML Schema component model creates some complexity in this area and we need to decide how to expose the information in the a usable way. That model makes a distinction between particles and terms, where the particle carries occurrence information. It also makes an analogous distinction between attribute uses and attribute declarations, where attribute uses carry value constraints. Attribute uses and particles also can be used to carry local context-specific annotations.
Figure 21: More Accessors
|
|
Return the particles within a model group or complex type's content type as values. |
|
|
Get the term for a particle. |
|
|
Return the attributes uses from a complex type. |
In addition to the new accessors listed in Figure 20,
the sc:attribute-decl accessor applies to attribute use component
values.
Figure 22: Declaration and Particle Values
Implementation Considerations
It is difficult to generalize about different implementations of a language. Overall implementing the accessors, type tests and operators discussed here on top of an existing XQuery implementation was not difficult.
XQuery is notoriously sensitive to syntactic extensions.
The use of functional syntax made the integration with an existing
XQuery lexer and parser more straight-forward than it would have been if new
syntactic constructs had been required. The new sequence type tests and the
subtype of operator did require additions to the lexer and parser
rules. The subtype of operator was added as an additional kind of
comparison expression; the new sequence type tests were added in parallel to
XPath kind tests. Adding the component kind tests required the same kind of
special handling already required for other names that appear in a similar
context, such as element or document-node.
The values themselves were implemented as classes providing faÇades over existing data structures used to implement XML Schema components. These data structures were already present to support validation and type assignment. An implementation that lacked such data structures would clearly find exposing types as values more challenging. It is unclear, however, whether a direct reflection of the schema component model is the right approach. It may be better to step back and design a cleaner API, and provide values over that model instead. Such a strategy would bring a higher implementation burden.
Applying types as functions depended on a prior foundation of XQuery 3.0 function items, and would be much more difficult to accomplish without that foundation. Given that foundation, applying simple types as functions is trivial, and applying complex types as functions only slightly more so. The existing infrastructure to support schema validation was a necessary prerequisite to applying complex types as functions, however.
The most challenging part of the implementation was providing access to annotations. The existing data structures did not preserve annotations as data model instances, as they were irrelevant for validation and type assignment and there was no existing API that provided access to the schema components. A surprising amount of work needed to be done in order to preserve annotations and expose them as nodes.
Finally, the experimental implementation actually did allow schema
component values to be added as children to data model instances, primarily for
debugging purposes. Adding a schema component value as a child added it as the
XML Schema serialization. In addition, fn:string was also
implemented over these values to provide a terse representation of them, again,
for debugging purposes. About half the implementation code went into these
debugging APIs.
Future Directions
More Accessors, Functions, and Operators
This paper has focused mainly on exposing types as values in XQuery with a few basic accessors, with some forays into some related schema components. Some meta-programming techniques rely on being able to examine the full richness of the schema component model — terms, particles, attribute uses, schema component affiliations, abstractness, and so on. The program to provide such full access is similar to what was outlined here: define type tests for the component, define linkage functions or accessors to get to that component, define accessors to get at the component properties, and define whatever other useful functions and operators seem appropriate.
More extensive additions to the syntax of XQuery to allow
variables to be used as the operand of instance of and cast
as expressions would improve usability further:
declare function my:filter-by-type($t as type(), $nodes as node()*)
{
for $n in $nodes where $n instance of $t return $n
};
Relationship to Schema Component Paths
Once we embark on the program of providing full access to all the schema components and their properties, the use of schema component paths arises. Schema component paths define an XPath-like syntax for describing and navigating W3C XML Schema XSD11.1 component models. Abbreviated syntax is defined for some path steps, and certain axes that skip over intermediate components can provide effective ways of navigating through content models, or selecting multiple components with a since path.
Schema component paths navigate through the schema component graph
of an assembled schema. As with XPath, each step in the path consists of an
axis combined with some kind of test and perhaps with a predicate. In schema
component paths, the test is a name test (possibly a wildcarded name test, and
possibly a test for an anonymous component), and the only predicate defined is
a positional predicate (selecting the Nth of like components). There are no
accessors for non-component properties: the axes act as accessor for component
properties, just like in XPath. About two dozen axes are defined, for each of
the component-to-component relationships in the graph. For example, the
annotation:: axis corresponds to the sc:annotations
accessor.
Figure 23: Some Schema Component Paths
/schemaElement::p:outer/type::0/schemaAttribute::p:inner
/p:outer/~0/@p:inner
/type::p:second/model::sequence/schemaElement::p:duplicate[2]/type::*
/~p:second/model::sequence/p:duplicate[2]/~*
Figure 23 shows some schema component paths
rooted at the component for the schema as a whole. The first path starts
at the root of the assembled schema (/). It then traverses the
schemaElement axis (schemaElement::) with a name test
(p:outer), selecting a global
element declaration with the name outer. The path continues
through the type axis (type::) with a name test (0)
that in this case matches a type definition with no name (0 being
the indicator for this case). The path to this point will select the locally
declared anonymous type of the element declaration 'outer'. Finally the path
concludes by traversing the schemaAttribute axis
(schemaAttribute::) with a name test (p:inner),
selecting an attribute declaration whose name is 'inner' within the anonymous
type definition. As a whole then, this path selects a particular attribute
declaration of a particular element declaration. The second path means the same
thing, but uses abbreviated syntax.
The third path again starts at the root of the assembled schema
(/). This time it traverses through the type axis
(type::) with a name test (p:second), thus selecting
a global type definition with the name 'second'. The path then traverses the
model axis (model::) with a name test (sequence),
thus selecting the model group in the content model of the type,
but only if it is a sequence. The path continues along the schemaElement axis
(schemaElement::) with a name test (p:duplicate) and
a positional constraint ([2]), thus selecting an element
declaration within the sequence that has a name of 'duplicate', but referring
to the second such element within the content model. Finally, the path
traverses the type axis (type::) with a
wildcard (*), selecting the type of the given element declaration,
whatever it might be. The fourth path means the same thing, but uses
abbreviated syntax.
How does this compare to the accessors we have defined above?
The accessors operate not on components but on
XQuery values. For example, sc:element-decl applies to nodes.
Some accessors, such as sc:type, apply to component values
also, and so behave somewhat like the schema component axes.
To get the full range of axes available in schema component paths, it would be
necessary to define an accessor function corresponding to each axis, and allow
it to apply to component values. To get the effect of the name tests, XPath
predicates must be applied to the results of the accessor. Some accessors
return simple properties of the components, such as its name. Schema component
paths do not provide for access to the non-component properties. Finally,
schema component paths use the slash (/) as a syntactic separator
between steps. Since schema components are not nodes, XQuery forbids using them
inside a path (although they can be the last step of a path). A FLWOR
expression, nested function call, or the XQuery 3.0 simple mapping operator
(!) must be used instead.
Table I
Summary of Comparison of Schema Component Accessors and Schema Component Paths
| Schema Component Paths | Schema Component Accessors |
|---|---|
| operate on schema components | operate on XQuery values |
| apply to schema in isolation | can link between date model items and schema component values |
| start at particular schema component, either root of schema or contextually supplied schema component | apply to particular item(), not rooted at top of schema |
| path step selects component-valued component properties only | acccessor selects various kinds of properties |
| steps combine axis with name test directly | component or name test separate from accessor |
| steps combined with slash | chains of accessors cannot be combined with slash: must nest function calls, use FLWOR expressions, or use XQuery 3.0 simple mapping operator instead |
| path syntax and semantics similar to XPath, but is not XPath | syntax and semantics are standard XPath |
Since schema component accessors do not apply to schemas in isolation, it is not possible to construct XQuery expressions that are exactly equivalent to the schema component paths given previously. However, we can give some expressions that are roughly equivalent, to give some sense of how they compare.
Figure 24: Rough Equivalents to Schema Component Paths
$context/sc:schema()!
sc:element-decl()[sc:name(.)=xs:QName('p:outer')]!
sc:type()[empty(sc:name(.))]!
sc:attribute-decl()[sc:name(.)=xs:QName('p:inner')]
$context/sc:schema()!
sc:type()[sc:name(.)=xs:QName('p:second')]!
sc:particles()[
typeswitch (sc:term(.))
case model-group(xs:sequence) return true()
default return false()]!
sc:particles()[
typeswitch (sc:term(.))
case particle(p:duplicate) return true()
default return false()][2]!
sc:type()
Figure 24 shows XQuery expressions using the
accessors that are roughly equivalent to the schema component paths shown in
Figure 23. These
expressions have to be grounded on some XQuery item ($context),
and it was necessary to invent a new accessor to link to the root of the
assembled schema governing the type information for that item
(sc:schema).
Schema component paths clearly express traversal through a schema component graph more compactly than the accessor and linkage functions do, in that they combine the accessor and the type test in one path step. In addition, schema component paths generally elide the distinction between terms and particles, which the chained accessors do not.
On the other hand, schema component paths do not provide a complete answer, however. They do not provide access to non-component information, nor do they bridge the boundary between schema components and nodes or other item types.
We could imagine allowing schema component paths to mix directly
with normal XPath paths to provide that linkage. Then a path such as
/book/chapter/title/type::*/annotation::*/xs:appinfo/my:special-stuff/@my:type
would switch back and forth between XPaths and schema component paths. Under
this scheme the type axis in the fourth step of the path would start a relative
schema component path with some component as the default. In this case,
the context is the element declaration component for the title
element.
Mixing schema component paths with normal XPaths directly in this
way is apt to be very confusing, however. The rules governing the slash are
very different. In addition, abbreviated schema component paths in this mixed
context confuses interpretation tremendously. Is the @my:type at
the end of the path a normal attribute node on the
my:special-stuff element or is it an attribute declaration?
Careful specification of the switching rules may eliminate ambiguities,
but it won't eliminate confusion for humans reading and writing such
paths. The similarity of schema component paths to XPaths that helps make them
more intuitive in isolation causes difficulty when used in the same context as
XPath. This confusion also makes implementation more difficult: higher levels
of analysis would be required to determine what kinds of operations are
allowable, or to optimize node paths properly.
Keeping schema component paths and XPaths clearly separated provides the power of both, without the confusion. Wrapping schema component paths within some kind of scoping expression and providing the rich set of accessor functions seems the best way to achieve this.
Figure 25: Example Integration of Schema Component Paths and Type Values
/book/chapter/title/sc:element-decl() !
scp {type::*/annotation::*}/
xs:appinfo/my:special-stuff/@my:type
Schema component paths are wrapped inside special syntactic for to keep them clearly separated from normal XPaths.
In summary, both schema component paths and the XQuery linkage and accessor functions both have their place. Schema component paths are suited to navigating through schema component graphs or pointing to specific schema components for schema analysis or reporting. The accessor functions are more useful to exposing the schema information relevant to a particular XQuery item in the context of some operations over data.
Summary and Conclusions
This paper has explored adding schema types as values into XQuery, providing some measure of structural type introspection. In an XQuery context, a type is an XML Schema type, and to provide even basic structural type introspection requires pulling in quite a bit of the complexity of the XML schema component model. Nevertheless, implementation need not be too challenging, and only modest changes to the XQuery data model and existing syntax are required. It is possible to take an incremental approach to adding additional capabilities, but even some basic abilities can provide significant coding power and flexibility.
Schema component paths can be used in combination with linkage function if they are integrated with care, to avoid confusion. Schema component paths satisfy use cases where deep analysis of the schema is required, while simple accessor and linkage functions work well for getting at basic metadata about the types of the data being operated on.
References
[C++] American National Standards Institute, 1430 Broadway, New York, NY 10018, USA. The ANSI C++ Standard (ISO/IEC 14882), 2002.
[Gabriel91] Richard P. Gabriel, Jon L. White, and Daniel G. Bobrow. Clos: Integrating Object-Oriented and Functional Programming. Communications of the ACM, 34(9):29–38, 1991. doi:https://doi.org/10.1145/114669.114671.
[Gabrysiak] Gregor Gabrysiak, Stefan Marr, and Falko Menge. Meta Programming and Reflection in PHP http://instantsvc.sourceforge.net/docs/metaprogramming-and-reflection-with-php-paper.pdf Accessed 2012-03-29.
[XSD11.1] W3C: Shudi (Sandy) Gao 高殊镝, C.M. Sperberg-McQueen, and Henry S. Thompson, editors. W3C XML Schema Definition Language (XSD) 1.1 Part 1: Structures . W3C, April 2012. http://www.w3.org/TR/xmlschema-11-1/
[SCD] W3C: Mary Holstege and Asir S. Vedamuthu, editors. W3C XML Schema Definition Language (XSD): Component Designators. Candidate Recommendation. W3C, January 2010. http://www.w3.org/TR/2010/CR-xmlschema-ref-20100119/
[F&O30] W3C: Michael Kay, editor. XPath and XQuery Functions and Operators 3.0 Last Call Working Draft. W3C, December 2011. http://www.w3.org/TR/xpath-functions-30/
[Java] Oracle Corporation. Java TM Platform Standard Ed. 7 API Documentation. http://docs.oracle.com/javase/7/docs/api/index.html. Accessed 2012-04-01.
[XSD11.2] W3C: David Peterson, Shudi (Sandy) Gao 高殊镝, Ashok Malhotra, C.M. Sperberg-McQueen, and Henry S. Thompson, editors. W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes. W3C. April 2012. http://www.w3.org/TR/xmlschema11-2/
[XQuery30] W3C: Jonathan Robie, Don Chamberlin, Michael Dyck, John Snelson, editors. XQuery 3.0: An XML Query Language Last Call Working Draft. W3C, December 2011. http://www.w3.org/TR/xquery-30/
[Singh04] Hermanpreet Singh. Introspective C++. Masters Thesis, Virginia Polytechnic Institute, 2004.
[Perl6] Larry Wall. Apocalypse 12: Objects; number 12 version 8. http://www.perl6.org/archive/doc/design/apo/A12.html May 2006. Accessed 2012-04-01.
[DM30] W3C: Norman Walsh, Anders Berglund, and John Snelson, editors. XQuery and XPath Data Model 3.0 Last Call Working Draft. W3C, December 2011. http://www.w3.org/TR/xpath-datamodel-30/