Balisage Paper: Two from Three (in XSLT)

The XSLT standards

Since its inception in the mid 90s, the world of XML has been governed by standards. Originally attempting to regularise the extension of web pages, XML was developed as a meta-syntax for markup, aimed at using a strict tree-based representation of propertied element nodes containing sub-trees or text nodes. Very soon work started on developing XSL as a formatted and paginated alternative to HTML for documents with professional appearance. As we’re all well aware, the two aspects of XSL, formatting and variable document generation, split into two orthogonal standards – XSL-FO and XSLT. The latter developed, with XPath (and associated XQuery), into a full declarative XML-transformational language. In most recent versions XSLT/XPath/XQuery have become full functional programming languages, with XML trees as central data type.

Significant work under the auspices of W3C has developed and finalised these standards for XSLT/XPath/XQuery^[2] through three major versions over the past 15 years. In each case a very comprehensive specification has been developed, reviewed, criticised and modified in cycles that are typically 3 years long and involve a few dozen contributors. Examples, test cases and experimental/operational implementations are all used to develop and finalise the specification, which is often followed by a further 3 years of polishing.

Once a standard (version) has been finalised, a degree of stability should then encourage developers of both implementations and applications to build and support software, without the language’s syntax or semantics altering. Of course it is exactly such full-scale use of a language that could expose shortcomings or new features that are needed to increase utility. The art of developing standards is to anticipate as much as is needed to get a useful and robust set of features that i) is adequate for significant application use but ii) not too complex for implementation or application.

XSLT 2.0 to XSLT 3.0

The major changes in XSLT (and XPath) between version 2.0 (2007) and version 3.0 (2014) are:

Why downgrade?

Two serious questions immediately arise:

The first point is the main instigator for this work and is described below. As far as the second point is concerned, code written in 3.0 can be more coherent than the 2.0 equivalents. Indeed in the description of xsl:iterate in the standard, it is noted that:

Moreover, the existence of a reasonable semi-automatic transformer can act as a stimulus to both developers and implementors, providing a means to write test and exemplar programs within 3.0 that themselves can be tested before full XSLT 3.0 processor ipeentations are available.

Implementation code for parts of the variable document architecture mentioned earlier had been upgraded to XSLT 3.0, to take advantage of the new features in increasing coherence, robustness and re-usability of the code. (For example many new parent-children layouts, such as all-children-in-a-circle , could be added merely by defining the XPath function required to work out the positions and place each component of a sequence – all encapsulation and child-recursive evaluation was handled in common. Similarly the addition of maps simplified a stack-frame implementation of single-assigment presentational variables.) As mentioned earlier, the author's variable document architecture was used principally to publish a set of instances of a variable data document bound to a sequence of data instances. As such it can be considered to be a server-side operation. In 2013 the release of Saxon-CE as a fully-compliant browser-based XSLT 2.0 implementation opened the possibility of looking at client-side and potentially interactive operation of the document architecture. However, by this point some significant sections of the implementation had migrated to XSLT 3.0, as well as altering or extending the semantics of some of the operations.

We now have the problem of two branches – the old 2.0 version, (which had not of course been updated to reflect changing semantics) and the now master 3.0 version. We could retrofit (and then develop both branches in parallel) but another option was worth exploring – could sections of the 3.0 code be converted to an operationally equivalent 2.0 code automatically? The answer is of course yes.

Assumptions

All this work reported here is based on the following assumptions:

XSLT constructs

Many of the additional XSLT instructions were targetted to support streaming but have other more general utility and may be susceptible to reversal in certain conditions. Those I have attempted so to engineer include:

<xsl:mode name="fred" on-no-match="shallow-copy" .../>

 <xsl:template
      match="@*|node()"
      mode="fred" priority="-1000">
      <xsl:copy>
         <xsl:apply-templates select="@*" mode="#current"/>
         <xsl:apply-templates select="node()" mode="#current"/>
      </xsl:copy>
   </xsl:template>

   <xsl:template match="/">
    <xsl:variable name="today" select="current-date()"/>
    <xsl:iterate select="//chapter">
      <xsl:param name="no" select="1"/>
      <xsl:copy>
        <xsl:attribute name="no" select="$no"/>
        <xsl:attribute name="date" select="$today"/>
        <xsl:sequence select="@*|*"/>
      </xsl:copy>
      <xsl:next-iteration>
        <xsl:with-param name="no" select="$no + 1"/>
      </xsl:next-iteration>
    </xsl:iterate>
  </xsl:template>

   <xsl:template name="d3e14.iterate-0">
      <xsl:param name="iterate.sequence"/>
      <xsl:param name="no"/>
      <xsl:param name="today" tunnel="yes"/>
      <xsl:variable name="head" select="$iterate.sequence[1]"/>
      <xsl:variable name="tail" select="subsequence($iterate.sequence,2)"/>
      <xsl:for-each select="$head">
         <xsl:copy>
            <xsl:attribute name="no" select="$no"/>
            <xsl:attribute name="date" select="$today"/>
            <xsl:sequence select="@*|*"/>
         </xsl:copy>
         <xsl:call-template name="d3e14.iterate-0">
            <xsl:with-param name="iterate.sequence" select="$tail"/>
            <xsl:with-param name="no" select="$no + 1"/>
         </xsl:call-template>
      </xsl:for-each>
   </xsl:template>
   
   ....
   <xsl:template match="/">
      <xsl:variable name="today" select="current-date()"/>
      <xsl:call-template name="d3e14.iterate-0">
         <xsl:with-param name="iterate.sequence" select="//chapter"/>
         <xsl:with-param name="no" select="1"/>
         <xsl:with-param name="today" select="$today" tunnel="yes"/>
      </xsl:call-template>
  </xsl:template>

<weekday>Today is {format-date(current-date(),'Fn')}</weekday>

<weekday>
  <xsl:value-of>
    <xsl:text>Today is </xsl:text>
    <xsl:value-of select="format-date(current-date(),'Fn')"/>
  </xsl:value-of>
</weekday>

XPath expressions

Whilst most focus has been on XSLT instructions, some XPath 3.0 expressions might also be susceptible to reversal in certain cases. In the most general case substitution will involve accurate and full parsing of an XPath expression to its parse tree. In restricted cases (that the developer could indicate), regular-expression substitution on the XPath expression strings might be sufficient. Here are some possibilities:

<xsl:value-of select="let $i := (1 to 6),
  $avg := avg($i) return $i[. gt $avg]"/>

<xsl:value-of>
  <xsl:variable name="i" select="1 to 6"/>
  <xsl:variable name="avg" select="avg($i)"/>
  <xsl:sequence select="$i[. gt $avg]"/>
</xsl:value-of>

map

Maps were again added to support dynamic collection and storage of data through a streamed processing operation. A common use of map{} is to associate a series of keys to values that are sequences of items (i.e. item()*). For example in the author's work on document layout a tunnelled variable $lay:variables as map(xs:string,item()*) contained bindings to named sections of layed-out components, that could be reused or examined (e.g. $lay:variables('background') might give all the items in the page background).

There are a number of possibilities for representing maps in certain (limited) circumstances. In the case where maps are isolated, i.e. attached to variables and not mixed with other items in a general sequence^[6], then a possible scheme is to emulate the map with a key-and-value stack of the following general form:

Figure 1: A map and its update

where the <binding/> elements are reserved forms and clearly distinguishable from normal content. On the left foo binds to a sequence of length 3, bar to one of length 2 and charlie to an empty sequence. After some update (i.e. an addition to the map in some more local context) shown on the right, foo now binds to a doubleton sequence, in this case result tree nodes that were bound to the variable $elements.

By searching for the last binding for a given key we can simulate map member updating, whilst still preserving locality of scope. (As all variable bindings in XSLT/XPath are single-assignment with variable name overriding strictly constrained to following-sibling::*/descendant-or-self::* situations, any local addition to a map is effectively copy-and-add which only has effect in the local scope.) The length attached to the binding is used to extract the appropriate number of values.^[7].

To implement map lookup for this scheme we substitute the interpolation $some.name(name) by a call to the (stylesheet-defined) function map:get($some-name,name)^[8] which looks up the last binding entry in $some-name for the given name (hence preserving locality of scope) and then returns the appropriate subsequence of the stack from immediately behind the binding element^[9].

This approach has some limitations that may be important depending upon the application. Most derive from the fact that the representation is not a single item (which a map(*) is) but a sequence. This means it cannot be treated as an item by some other manipulation – it will produce incorrect results for functions and operators that do inspect items, such as count(), is and instance of^[10]. The function empty() will be unable to distinguish between an empty map and no map (i.e. exists($map) and map:size($map) = 0 would be indistinguishable from empty($map)). Equally well this scheme cannot represent a sequence of maps. In the author's limited experience most conventional use of maps rarely involves such existential manipulation of arbitrary maps. Perhaps mostly simply, this scheme only works for interpolations of keyed-values from maps.

An alternative method, which separates the stack from the keys, requires binding to two separate XSLT variables^[11] e.g. xsl:variable name="name" as="map(*) → xsl:variable name="name.stack" as="item()*, xsl:variable name="name.keys" as="element(map:binding)*. Now the bindings contain an offset and length into the stack. This approach can be more efficient in lookup, but is significantly more restricted in the situations in which it can be deployed.

Maps in XSLT 3.0 can be created through three possible mechanisms: XPath syntax (map{ key : value,...})^[12], XPath constructor functions (map:entry(), map:merge()) or XSLT instructions (xsl:map, xsl:map-entry). How and to what extent, these features might be simulated in 2.0 is discussed later.

Arrays, being introduced in XPath3.1, which can store mixed arrays of items (including sequences of items), could be emulated in a similar manner, using a reserved marker element between array members, and defining emulations of the constructor and accessor functions (array:get() etc.). Restrictions on existential manipulation of these emulations are similar to those for map().

Compile-time modification

Some features are aimed at evaluating or modifying the XSLT statically at compile time, with a @static property being supported on global variables and parameters. For many of the situations envisaged, the 2.0 program will be projected from the master 3.0 version under conditions which are effectively static. Thus many of these components may be pre-evaluated. In some cases, where the static variables are used to control conditional code use, this might be essential.

As an example default push-mode behaviour for a given mode might be selected statically through a toggle and a mutually exclusive pair of declarations:

<xsl:variable name="process-all" select="false()" static="yes"/>
<xsl:mode name="process" on-no-match="shallow-copy" use-when="$process-all"/>
<xsl:mode name="process" on-no-match="deep-copy" use-when="not($process-all)"/>

Clearly, provided XPath expressions can be parsed and evaluated statically, this choice can be predetermined. (In fact it has to, otherwise conflicting templates at the same priority will be placed in the resulting stylesheet.) In a similar manner, XSLT provides shadow attributes whose values can be computed statically. In this way the choice above could be rephrased as:

<xsl:variable name="process-all" select="false()" static="yes"/>
<xsl:mode name="process2" _on-no-match="{if($process-all) then 'deep' else 'shallow'}-copy"/>

which is similarly capable of being evaluated during the program transformation.

Obviously to perform this evaluation we need to be able to evaluate XPath expressions with associated variable bindings dynamically. Within XSLT3.0 this can be acheived through the xsl:evaluate instruction and is discussed further in section “Conditional compilation”.

Converting xsl:mode

Transforming the xsl:mode instruction is relatively straightforward. A template matching an xsl:mode defining no-match behaviour has conditional choices of content, derived effectively from XSLT3.0 Built-in Template Rules, chosen dependent upon the @on-no-match attribute value:

<xsl:template match="xsl:mode[@on-no-match]"
  <xsl:param name="priority" select="-1000"/>
  <xsl:variable name="node.patt">*|text()|comment()|processing-instruction()</xsl:variable>
  <xsl:choose>
    <xsl:when test="@on-no-match='text-only-copy'"/>
    <xsl:otherwise>
      <xsl:choose>
        <xsl:when test="@on-no-match='shallow-copy'">
          <X:template match="@*|{$node.patt}|document-node()" mode="{@name}" priority="{$priority}">
            <X:copy>
              <X:apply-templates select="@*" mode="#current"/>
              <X:apply-templates select="node()" mode="#current"/>
            </X:copy>
          </X:template>
        </xsl:when>
        <xsl:when test="@on-no-match='deep-copy'">
          <X:template match="@*|{$node.patt}|document-node()" mode="{@name}" priority="{$priority}">
            <X:copy-of select="."/>
          </X:template>
        </xsl:when>
         ......
      </xsl:choose>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

The prefix X: is bound to an aliased namespace that will become xsl: in the output. The priority is chosen, as remarked earlier, to be lower than any priority found for that mode within the entire expanded stylesheet, but higher than the (text-only-copy) default rules. (It should be possible to find a reasonable minimum priority through min(xsl:template/@priority) - 1.) Text-only-copy behaviour is the built-in default, so a request for that requires no substituted templates. (More strictly, these defaults should be placed in the lowest precedence position, which means in a separate stylesheet itself imported at lowest precedence, i.e. as the first import of the toplevel stylesheet.)

Converting xsl:iterate

Processing the xsl:iterate instruction into the recursive template declaration and xsl:call-template instruction pair shown earlier is rather more complex and needs to be a recursive process as of course the bodies of iterations can themselves contain further iterations.

Luckily using push-template programming supports such recursion naturally. The process has two parts: formation of the templates (as top-level declarations in the result stylesheet) for each of the iterate instructions found, and generation of the specific call.

There are several specific aspects of the xsl:iterate instruction that must be considered. Firstly, the only result(s) of the instruction arise from the evaluation of the sequence constructors contained within. Consequently these sequence constructors need to be preserved in the resulting code. Secondly, the iterative nature of the instruction is performed by xsl:next-iteration directives which potentially modify transmitted (state) information through parameters – these will be converted into suitably parameterised xsl:call-template instructions. Thirdly, extraordinary exit and postlude processing needs to be supported. Finally, the whole xsl:iterate instruction operates within a local scope within which variables can exist and whose values can be interpolated. Any solution must preserve such bindings, including local bindings within nested iteration constructs.

The transformation process involves the following stages:

Converting text value templates

Interpolation of text value templates is controlled by @[xsl:]expand-text attributes attached to ancestor elements. Processing these is straightforward, with a pre-emptive template setting a boolean state, and text nodes which contain such text value templates being processed by regular expression analysis:

<xsl:template match="*[@xsl:expand-text]|xsl:*[@expand-text]" priority="6">
  <xsl:next-match>
    <xsl:with-param name="expand-text" as="xs:boolean" tunnel="yes"
         select="(@expand-text|@xsl:expand-text) = ('yes','true')"/>
  </xsl:next-match>
</xsl:template>
<xsl:template match="*/@xsl:expand-text|xsl:*/@expand-text" priority="6"/>
<xsl:template match="text()[contains(.,'{')]">
  <xsl:param name="expand-text" as="xs:boolean" select="false()" tunnel="yes"/>
  <xsl:choose>
    <xsl:when test="$expand-text">
      <X:value-of>
        <xsl:analyze-string select="." regex="{$reg.AVT}" expand-text="yes">
          <xsl:matching-substring>
            <X:value-of select="{regex-group(1)}"/>
          </xsl:matching-substring>
          <xsl:non-matching-substring>
            <X:text>{.}</X:text>
          </xsl:non-matching-substring>
        </xsl:analyze-string>
      </X:value-of>
    </xsl:when>
    <xsl:otherwise>
      <xsl:next-match/>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template> 

Converting map{}

The major issue is whether a full XPath expression parsing is required. As hinted previously, with some (oracle-provided) guarantees on the content of string literals (or more accurately the absence of certain forms in such strings), simpler mechanisms using regular expressions may suffice. The most important is that the regular expression \$\i\c*\((\i\c*\)) can recognise such map interpolations.

The model for map equivalents suggested earlier, using a stackframe, requires two different actions: interpolation from the map stack and adding values to the stack. As described earlier a map interpolation can be replaced with a function call referencing the stack frame: $some-name(name) → map:get($some-name,name). One implication of this approach is that a map cannot be a value in another map, i.e. we cannot support anonymous maps – they must all have (variable) names within XSLT/XPath scope.

How do we build the map construction features? The main technique is to try to convert other constructors to a call on emulations of the map construction functions, such as map:entry($key,$value), map:merge($maps) and so forth.

When the value of the map is defined by XSLT instructions such as:

<xsl:variable name="map1" as="map(*)">
  <xsl:map>
    <xsl:map-entry key="'one'" select="'First'"/>
    <xsl:map-entry key="'rest'" select="'Second','Third','Fourth'"/>
  </xsl:map>
</xsl:variable>
<xsl:sequence select="'1:',$map1('one')"/>
<xsl:variable name="map1" as="map(*)">
  <xsl:map>
    <xsl:sequence select="$map1"/>
    <xsl:map-entry key="'foo'" select="1,2,3,4,5"/>
    <xsl:map-entry key="'one'" select="'Primus','inter','pares'"/>
  </xsl:map>
</xsl:variable>
<xsl:sequence select="'2:',$map1('one')"/> 

then a push-template implementation along the following lines:

<xsl:template match="xsl:variable[@as='map(*)'][*]">
  <xsl:copy>
    <xsl:sequence select="@name"/>
    <xsl:attribute name="as">item()*</xsl:attribute>
    <xsl:apply-templates select="xsl:map" mode="#current"/>
  </xsl:copy>
</xsl:template>
<xsl:template match="xsl:map">
  <xsl:apply-templates select="*" mode="#current"/>
</xsl:template>
<xsl:template match="xsl:map-entry[@select]">
  <X:sequence select="map:entry(({@key}),({@select}))"/>
</xsl:template> 

produces a working form with suitable constructs:

<xsl:variable name="map1" as="item()*">
  <xsl:sequence select="map:entry(('one'),('First'))"/>
  <xsl:sequence select="map:entry(('rest'),('Second','Third','Fourth'))"/>
</xsl:variable>
<xsl:sequence select="'1:',map:get($map1,'one')"/> → 1: First
<xsl:variable name="map1" as="item()*">
  <xsl:sequence select="$map1"/>
  <xsl:sequence select="map:entry(('foo'),(1,2,3,4,5))"/>
  <xsl:sequence select="map:entry(('one'),('Primus','inter','pares'))"/>
</xsl:variable>
<xsl:sequence select="'2:',map:get($map1,'one')"/> → 2: Primus inter pares 

When maps are built with calls to the map constructor functions (e.g. map:entry()) then by providing suitable emulation functions in a stylesheet-defined library, XPath expressions need not be touched. Some of the emulation functions are as follows:

<xsl:function name="map:entry" as="item()+">
  <xsl:param name="key" as="xs:anyAtomicType"/>
  <xsl:param name="value" as="item()*"/>
  <xsl:sequence select="$value"/>
  <binding name="{$key}" length="{count($value)}"/>
</xsl:function>
    
<xsl:function name="map:merge" as="item()*">
  <xsl:param name="maps" as="item()*"/>
  <xsl:sequence select="$maps"/>
</xsl:function>

<xsl:function name="map:keys" as="xs:anyAtomicType*">
  <xsl:param name="maps" as="item()*"/>
  <xsl:sequence select="distinct-values($maps[self::binding]/@key)"/>
</xsl:function> 

(Since our map emulation is merely a sequence, a sequence of such maps is already a map, so map:merge() just passes through). Using these functions as interfaces decouples the exact details of the representation from the actual use of maps.

When maps are defined in XPath map syntax map{ }, we need to modify the XPath parse tree. Ignoring tokens, the tree for map{ 1 := ('fred',3+4), 2 := 'bert'} is shown on the left:

<MapExpr>
  <Literal type="xs:integer" value="1"/>
  <Expr>
    <Literal type="xs:string" value="'fred'"/>
    <AdditiveExpr op="+">
      <Literal type="xs:integer" value="3"/>
      <Literal type="xs:integer" value="4"/>
    </AdditiveExpr>
  </Expr>
  <Literal type="xs:integer" value="2"/>
  <Literal type="xs:string" value="'bert'"/>
</MapExpr> 

<Expr>
  <FunctionCall name="map:entry">
    <Literal type="xs:integer" value="1"/>
    <Expr>
      <Literal type="xs:string" value="'fred'"/>
      <AdditiveExpr op="+">
        <Literal type="xs:integer" value="3"/>
        <Literal type="xs:integer" value="4"/>
      </AdditiveExpr>
    </Expr>
  </FunctionCall>
  <FunctionCall name="map:entry">
    <Literal type="xs:integer" value="2"/>
    <Literal type="xs:string" value="'bert'"/>
  </FunctionCall>
</Expr> 

so if we surround each pair of children with a function call to map:entry() and replace the MapExpr element with Expr, we get the tree on the right that would have been parsed from (map:entry(1,('fred',3+4)), map:entry(2,'bert')), which is the desired representation for the initialised map. Back-conversion of the parse tree into text correctly modifies the XPath expression. (If the representation of the map is something other than a sequence of map:entry() results, such as using map-start...map-end markers, then surrounding this construct with a function call to map:merge() will be adequate.)

Conditional compilation

To support conditional compilation (either through the @use-when directive or within shadow attributes), we need two activities: firstly to collect and detemine the values of all the (global) static variables and secondly to evaluate the XPath expression of the @use-when or attribute value templates within shadow attributes, using those bindings. It is possible for the values of static variables to be interpolated within @use-when directives (which may be attached to static variable declarations) or shadow attributes, so these two processes must be handled concurrently.

For both cases the most complex is to determine the values of the static variables. Luckily they are defined to have their values determined only by @select attributes (i.e. XPath expressions, no XSLT instructions can influence) and reference between static variables (which must all be top-level children of stylesheets and have unique names) is only permitted in a reverse direction. Hence an iteration across the top-level children of a stylesheet, evaluating any static variables with possible variable bindings already accumulated into a map, evaluating the effect of static variables on the top-level trees and determining @use-when effects, will produce the statically evaluated top-level stylesheet children:

<xsl:template match="xsl:stylesheet|xsl:transform" mode="X:static" as="element()">
  <xsl:copy>
    <xsl:sequence select="@*"/>
    <xsl:iterate select="*">
      <xsl:param name="vars" as="map(*)" select="map{}"/>
      <xsl:variable name="conditional" as="element()?">
        <xsl:apply-templates select="." mode="#current">
          <xsl:with-param name="static.var.values" select="$vars" tunnel="yes"/>
        </xsl:apply-templates>
      </xsl:variable>
      <xsl:choose>
        <xsl:when test="$conditional/self::xsl:variable[@static='yes']">
          <xsl:variable name="value" as="item()*">
             <xsl:evaluate xpath="@select" with-params="$vars"/>
          </xsl:variable>
          <xsl:copy>
            <xsl:sequence select="@* except (@static|@use-when)"/>
            <xsl:attribute name="select" select="X:select($value)"/>
          </xsl:copy>
          <xsl:next-iteration>
            <xsl:with-param name="vars" select="map:merge(($vars,map{@name := $value}))"/>
          </xsl:next-iteration>
        </xsl:when>
        <xsl:otherwise>
          <xsl:sequence select="$conditional"/>
          <xsl:next-iteration/>
        </xsl:otherwise>
      </xsl:choose>
    </xsl:iterate>
  </xsl:copy>
</xsl:template> 

(The function X:select() converts atomic items into suitable XPath expression values, such as surrounding xs:string with quotes.) Toplevel children are processed with matching templates for @use-when and shadow attributes^[17]:

<xsl:template match="xsl:*[@use-when] | *[@xsl:use-when]" mode="X:static" priority="5">
  <xsl:param name="static.var.values" tunnel="yes"/>
  <xsl:variable name="use-when" as="xs:boolean">
    <xsl:evaluate xpath="@use-when|@xsl:use-when" with-params="$static.var.values"/>
  </xsl:variable>
  <xsl:if test="$use-when">
    <xsl:next-match/>
  </xsl:if>
</xsl:template> 

During this phase it is also possible to project the consequences of static information that will be true in the execution context of the transformed stylesheet. For example the built-in function system-property(property-name) can yield information about the implementation, such as the version of XSLT supported. This is often used within conditional code, such as use-when="system-property('xsl:version') = '3.0'". In this case we anticipate that the transformed stylesheet will operate under a regime where system-property('xsl:version') has the value '2.0', so if we replace such a function call with its expected value, the conditionality can be projected during the transformation.

Can it process itself?

I stated earlier that the converter code was written in XSLT 3.0. This of course begs the question whether it can generate a 2.0 version of itself. Obviously this depends upon the complexity of the code written. At the time of writing, the some 1500 lines appear to use only the following 3.0 features:

Of these features, string concatenation would require full XPath parsing, as it uses operator associativity extensively and is thus very unsuitable for regular expression handling. (A refactoring to use concat() is probably overdue – actually they aren't used in very complex situations.) The use of map{} is comparatively simple, but does involve functional constructors. The most thorny issue is use of xsl:evaluate. In (commercial) Saxon 2.0 implementations there is a saxon:evaluate() function that can be be exploited, with suitable variable bindings as discussed earlier. Unfortunately this isn't a option in the current open-source Saxon-CE^[20].

Any system involving program generating program inevitably encounters quoting problems - how to distinguish real program from program to be written. In XSLT the xsl:namespace-alias declaration helps solve this issue, in this case by declaring the X: prefixed namespace to map into the XSLT namespace in the final result. However if the compiler is processing itself, then input elements marked with that prefix need to be preserved. This requires a remapping and re-alisassing.

Full XPath parsing was added to the repertoire and the the system tuned by examining it operating on itself. The first goal was of course to generate a result that will compile successfully. Having reached that point, further tuning was required to get successful operation of the generated compiler on small tests, whose runs could be compared against benchmarks.

If this transformation is successful then of course the resulting 2.0 version of the converter should again be capable of converting its 3.0 master copy into 2.0, producing an identical copy of the converting stylesheet. Thus if Convert3 is the converting stylesheet, then

let Convert2 :=  Convert3(Convert3);
    Convert2(Convert3) == Convert2

should yield the given equality.The author can report that this is now indeed the case^[21]. Stability and idempotency has been tested successfully through checking that

Convert2(Convert3) = (Convert2(Convert3))(Convert3)

Alternatively this is best shown through the following diagram, where the final two generated versions of the converter are found to be identical:

Figure 6: Conversion of the XSLT3to2 converter to XSLT2.0

Further possibilities

The list of assumptions (section “Assumptions”) excluded a number of areas that might be interesting to explore. Streaming could be treated as a transparent operation, but of course there would be no performance guarantees and unless full streamability analysis was undertaken, programs that would ordinarily be deemed unstreamable would still operate. Simple packaging based on static analysis, flattening, visibility projection and overriding replacement is a distinct possibility: provided all stylesheet packages are available, the package processing is effectively a static operation ^[22] .

The most intriguing would be to investigate what sort of function-valued item behaviour could be supported, by attempting to attach to Dimitre Novatchev's FXSL framework (Novatchev2006). Most of the standard higher-order functions have implementations within the library, and static definitions of function items can be determined from XPath parses, so the issue is whether these can be converted to appropriate template-reference elements and associated template and function trios. Whilst this paper has described a useful model for maps in some circumstances, a system that could accomodate dynamic functions may support maps more accurately, as essentially they are functions over a finite discrete domain of key values.

Acknowledgements

Most of the impetus for this work came from the work of Michael Kay, Philip Fearon and O'Neil Delpratt when they released SaxonCE in 2011. Offering a possibility of trying my document system directly in a browser environment acted as the stimulus to looking at detailed program transformation of the by-then-extensive codebase. Little did I realise that a few years later I'd be working with them.