Balisage Paper: XTemp: Event-driven Testing and Monitoring of Business processes

Objectives and Requirements

XTemp is an acronym for 'XML Testing and Event-driven Monitoring of [business] Processes' also suggesting a strong 'temporal' aspect. The monitoring and testing of business processes and transactions serves anyone or a combination of these objectives:

In all these cases, it is assumed a business process generates a trace of its behavior - either as part of its business communicating function (messages) or as a log of its activities. It is also assumed that this trace can be modeled as events of various kinds, that are represented in - or may be mapped into - an XML format. This event format may be either a proprietary vocabulary or may follow a standard such as XES (http://www.xes-standard.org/), SAF (Symproms Automation Framework from OASIS), CEE (http://cee.mitre.org/), DMTF/CIM or XDAS. For example, a trace of exchanges between a Web client application and a Web service can be formatted in XML as done in [WSI10], including HTTP headers.

More detailed XTemp requirements are:

General Positioning of XTemp

XTemp is workflow-capable in the sense that it provides a minimal set of operators that enable most workflow patterns, of which a good account has been provided in [RHAM06] (see also http://www.workflowpatterns.com/). Compared with workflow languages that are also XML-based and support a notion of event (BPMN / XPDL, WS-BPEL), XTemp has more of an analytical focus. While BPMN and XPDL have restricted the use of events to include only those types of events that will affect the sequence or timing of activities of a process, XTemp is accessing past events for correlation and analysis. Consequently its concept of Event Board is more than an event Source, but also a queryable repository of events that can be used off-line or live.

Because its focus is the analysis of processes and of transactions via the events they generate, the general execution model of XTemp is time-aware and event-centric. Time awareness is based on a notion of virtual present date/time (VP-time) associated with a script execution, and decoupled from the actual present time (AP-time). The VP-time which can be set as part of the execution context - i.e. independently from the script logic - allows for the same script to execute either in a live mode (consuming events as they occur) or deferred mode (i.e. for static analysis of past events) or yet in a way that combines consumption of both real-time and logged events. As explained in the execution model overview, concurrency in XTemp is based on the notion of VP-time.

XTemp may be viewed as “yet another concurrent programming language”, but it is the first one that leverages XML in three ways: (a) for input (event) processing, (b) for in-line report generation, and ( c) for its own syntax which allows for generating scripts from script templates using transforms. Conversely, XTemp could be defined as “yet another XML processing language” close to XSLT, but one with built-in support for event processing support and concurrency capability. How XTemp relates to XSLT is discussed in more details in the Related Works section.

Event Processing

Because events often reflect on concurrent, distributed processes, XTemp supports a form of concurrency. However it is not intended for general concurrent programming and lacks many of the features found in such languages (e.g. advanced execution control as in coroutines, resource access control as semaphores or mutex). There are however two features that distinguish XTemp and highlight its analytic focus:

The notion of VP-time combined with the access allowed by event boards makes it possible to synchronize script execution with events regardless of the actual time of their occurrence. This in turn allows for executing the same script 'live' or 'deferred' (i.e. for static analysis of past events) by simply modifying the initial VP-time value.

Time Control

An XTemp script is composed of several script units called 'scriplets' that can invoke each other, with functional semantics with regard to the output they may or may not produce - XML fragments called 'X-effect' - the aggregation of which represents the complete script output called 'report'. Scriplet execution follows a (virtual) thread model: scriplets may be concurrent to each other if their VP-times overlap, but more important is how they influence each other's VP-times. By default, a scriplet invocation is 'synchronized' as it is blocking the invoking scriplet ('blocking' invocation) until completion of the invoked scriplet. This blocking means that the next statement after the invocation is set to the ending VP-time of the invoked scriplet - in which case both scriplets can still execute in a single thread implementation-wise as they cannot consume (or produce) same or co-occurring events. A 'non-blocking' scriplet invocation will decouple the VP-times and allow both scriplets to access same or co-occurring events, with the effect of a parallel execution more aligned with a conventional and stronger meaning of concurrency.

Even for static analysis over a log of past events, it may be convenient and more intuitive to script for concurrent executions. As illustrated later, a scriplet specialized for capturing a particular sequence of events independently from the main execution thread (i.e. represented by the invoking scriplet), should be scripted as fully concurrent to the invoking scriplet, whether the execution is live or not.

XTemp concurrency features are minimal but sufficient to track (either live or deferred) most workflow execution patterns. Non-blocking scriplet invocation allows for matching conventional workflow forking (or 'and-split'). The merging (or 'and-join') of one or more concurrent workflow paths is supported in XTemp by using a single primitive ('wait') that allows for synchronizing with all kinds of clock events - e.g. a date or a scriplet completion.

In addition to the scriplet invocation mode (blocking vs. non-blocking), the VP-time of a scriplet execution may be set at its invocation: a VP-time setting anterior to the VP-time of the invoking scriplet is called a backward invocation, while a VP-time setting future to the VP-time of the invoking scriplet is called a forward invocation. The invocation mode combined with VP-time setting allows for convenient expression of the logic behind various patterns of events:

In terms of concurrency semantics, XTemp cannot be considered an Actor language: scriplets are not aware of each other and do not communicate explicitly with each other. The model is rather of CSP (Communicating Sequential Processes), where scriplets behave as processes, consuming events and communicating via events only - except when a scriplet is starting another scriplet, in which case parameters may be passed. When scriplets do not communicate with each other e.g. by posting events, a single-threaded implementation of XTemp (such as most XSLT engines provide) is sufficient. When intended for static analysis of complete event boards (e.g. event logs), the subset of XTemp language identifed as 'core' - which excludes posting of new events - can be implemented in XSLT using conventional XSLT engines.

General Execution Paradigm

An XTemp script is consuming events – either in real time or from a log. XTemp is not entirely event-driven though: a script makes use of conventional algorithmic statements (branching, loops, exits, serial or concurrent sub-script invocations) in order to control the flow of execution of testing/monitoring units called “scriplets”. These scriplets may (or may not) synchronize with events. This combination of workflow-style control (allowing concurrency) and event synchronization is most appropriate for process monitoring and the analysis of related event patterns. Scriplets provide a parameterized, reusable unit of execution and are assigned a date/time (their “virtual present time”) when executing. The script - or script package - is the largest unit of execution. The binding between the script and an actual event source (event board), is defined inside an “execution context” element separate from the script logic.

The XML practitioner take-away: XTemp has constructs close to XSLT constructs, however its execution is not driven by XML patterns or templates. Its controls are workflow-like with a built-in concept of time and event, as it was designed to be familiar to business process designers and programmers. Yet, the “core” conformance level of XTemp is implementable in XSLT2.0 or above, as this was a key requirement. It is also designed for the XML practitioner: XPath has a central role for expression logic and event selection, and inline XML fragments of foreign namespaces allow for an intuitive generation of XML reports – in a way similar as XSLT does.

EXAMPLE 1: Periodic tallying of Purchase Order messages

The abbreviated monitoring script below illustrates some features of XTemp.

<script-package 
...
		xmlns="http://docs.oasis-open.org/tamie/xtemp/200909"
		xmlns:myreport="http://www.abc.com/TBD/report"
		xmlns:xs="http://www.w3.org/2001/XMLSchema"
		name="CUC-2">

  <execution-context>
    <start-with scriplet="mymainloop"/> <!-- starting point -->
    <!-- global variables -->
    <var name="maxiter" type="integer">5</var> 
    <var name="initdate" type="dateTime">2009-10-08T11:12:00</var> 
    <var name="period" type="duration" expr="xs:dayTimeDuration('P1DT1H')"/> <!-- 1 day + 1 hour -->
    <event-board name="POlog" event-store="myeventlog" mode="source"/>
  </execution-context>

  <!-- ======= scriplet: mymainloop ======= -->
 
  <scriplet name="mymainloop">
    <myreport:monitoringreport period="{$period}"> 

      <loop>
      …
	<until expr="$mycounter ge $maxiter"/>
	<var name="currentevts">
	  <start scriplet="doperiod" vptsync="true" vptset="$currentdate">
            <with-param name="startperiod" expr="$currentdate"/>
            <with-param name="lengthperiod" expr="$period"/>
	  </start>
	</var>
      …
	<myreport:currentevts period="{$mycounter}" startdate="{$currentdate}">
	  <eval expr="$currentevts"/>
        </myreport:currentevts>
	<lvar-next name="mycounter" expr="$mycounter + 1 "/>
	<lvar-next name="currentdate" expr="($currentdate + $period)"/>
	...
      </loop>
    </myreport:monitoringreport>
  </scriplet> <!-- mymainloop -->

  <!-- ======= scriplet: doperiod ======= -->
  <!-- The scriplet "doperiod" gathers various metrics for the period: 
average response time of Purchase Orders, total amount ordered, etc.  -->

</script-package>
	

  <!-- ======= scriplet: doperiod ======= -->

  <scriplet name="doperiod">
    <param name="startperiod" type="dateTime"/>
    <param name="lengthperiod" type="duration"/>

    <var name="endperiod" type="dateTime">xsd:dateTime($startperiod + $lengthperiod)</var>
    <loop>
      ...
	<var name="po-event">
          <catch eb=”POlog” vptend=”{$endperiod}”>
            <match>
	      <condition>xtemp:content//myapp:PurchaseOrder</condition>
            </match>
          </catch>
	</var>
	<until expr="not($po-event/xtemp:event)"/>

      <lvar-next name="count1" expr="$count1 + 1"/>
      ...
    </loop>
  </scriplet> <!-- doperiod -->
	

An example of generated report from running this script over a sample event board is:

<myreport:monitoringreport       
  xmlns:xtemp="http://docs.oasis-open.org/tamie/xtemp/200909"
  xmlns:myreport="http://www.abc.com/TBD/report"
  ...
  period="P1DT1H">
   <myreport:currentevts period="0" startdate="2009-10-08T11:12:00" >2</myreport:currentevts>
   <myreport:currentevts period="1" startdate="2009-10-09T12:12:00" >0</myreport:currentevts>
   <myreport:currentevts period="2" startdate="2009-10-10T13:12:00" >0</myreport:currentevts>
   <myreport:currentevts period="3" startdate="2009-10-11T14:12:00" >1</myreport:currentevts>
   <myreport:currentevts period="4" startdate="2009-10-12T15:12:00" >2</myreport:currentevts>
   <myreport:finaldate>2009-10-13T16:12:00</myreport:finaldate>
   <myreport:elapsedtime>P5DT5H</myreport:elapsedtime>
</myreport:monitoringreport>
	

Some Convenient Algorithmic Constructs

Three features are particularly helpful for testing and monitoring scripts:

Using both regular variables and mutable loop variables, is illustrated in the loop example below:

<loop>
  <on-start>  <!-- some mutable variables -->
    <lvar name="myitems" expr="$PO/lineItems/item"/>
    <lvar name="item" type="int">1</lvar>
    <lvar name="pototal" type="int">0</lvar>
  </on-start>
  <until expr="$item gt fn:count($myitems)"/>
  <var name="myItemAmount">
    <eval expr="$myitems[$item]/unitprice * $myitems[$item]/quantity"/>
  </var>
  <var name="itemRebate">
    <decide>
      <if expr="$myItemAmount gt 1000">
        <eval expr="$myItemAmount * 0.10"/>
      </if>
      <else>
        <eval expr="0"/>
      </else>
    </decide>
  </var>
  <lvar-next name="pototal" expr="$pototal + ($myItemAmount -
$itemRebate) "/>
  <lvar-next name="item" expr="$item + 1"/>
  <on-final>
    <abc:mypototal><eval expr="$pototal"/></abc:mypototal>
  </on-final>
</loop>
	

The XML practitioner take-away: Variables can be made mutable only where this capability is most needed: in loops (loop variables). They are distinguished from regular (immutable) variables by a different declaration syntax ( <lvar> vs. <var>), and are otherwise referenced similarly. It was acceptable to severely restrict the scope of mutable variables (local to a loop) and their mutation (only at the end of each iteration using <lvar-next>), while these restrictions made an XSLT implementation possible. An initialization loop section <on-start> inside the loop statement was then necessary to initiate their state, while a closing section <on-final> inside the loop was needed to extract the final values of mutable variables e.g. to report it as part of the X-effect.

Event Processing

A single statement <catch> is controlling event input, either for catching a single event or a pattern of events. An event pattern is a set of events correlated based on content, and often following some time-ordering constraint or time window constraint. An event pattern may be a well-correlated conversation such as a multi-message eBusiness transaction, or a more loosely defined co-occurrence of events e.g. close to each other in time but without well-established causality or common cause.

In XTemp an event pattern may be expressed in two ways: (a) by using one catch statement for each event, and by controlling the sequence(s) of <catch> statements using other XTemp controls (scriplets, conditionals, loops), or (b) by using a combination of filters (<match> statements) inside a single catch. The former (a) is more appropriate when the pattern occurs over a significant time window (such as for some long-lasting business transactions) or when the pattern is itself defined dynamically depending on some event content, requiring some processing as it unfolds (e.g. when a business transaction may unfold differently based on its initial event content). The latter (b) allows for a compact expression of deterministic patterns but more importantly allows for delegating the <catch> execution to specialized event processors that can optimize such processing (equivalent of query optimization in databases), e.g. by delegating to a complex event processing (CEP) module.

EXAMPLE 2: Catching some event pattern

The following catch statement selects a sequence of three events representing an exchange of three messages:

These three events are ordered in time: E3 after E2 after E1. The correlation between these events is based on different content:

<catch>
  <match event="E1">
    <condition>content/soap/Header/msgData/action = 'PORequest'</condition>
  </match>
  <match event="E2" after="E1">
    <condition>(content/soap/Header/msgData/action = 'POAccept' | 'POReject') and 
content/soap/Header/msgData/property[@name = 'order_ref'] = 
$E1/content/soap/Header/msgData/property[@name = 'order_ref']</condition>
  </match>
  <match event="E3" after="E2">
    <condition>content/soap/Header/msgData/action = 'Receipt' and 
content/soap/Header/msgData/property/RefMessageID = $E2/content/soap/Header/msgData/messageID
    </condition>
  </match>
</catch>
	

<catch> plays two roles: (1) synchronization operation for real-time execution, and (2) event board querying for past events analysis (deferred processing). In both cases, <catch> is operating using an event board as event source: the only difference between live and deferred is the VP-time of execution compared with the timestamp of events in board. “Catching” an event pattern 'live' often involves a combination of real-time event consumption with querying past events.

In contrast, WS-BPEL event catching <pick> is designed to capture and process single events at a time, as its use of events is more as a process driver, than for analytical and diagnosis purpose.

The XML practitioner take-away:Xpath2.0 proved sufficient as expression language for event querying, selection and correlation. Because event pattern expressed in <catch> are usually well time-constrained and/or strongly correlated, large logs of (time-ordered) events are manageable without penalty using conventional XPath and XSLT processors. Any XML-based event format - e.g. XES, SAF (Symproms Automation Framework from OASIS), CIM Events/Indications from DMTF - can be accommodated, provided that these events are timestamped so they can be ordered by the event board. In the baseline “core” implementation of XTemp with XSLT2.0, the event log is a plain file of time-ordered event records. Logs of events that have intermediate structures – e.g. events are grouped in “traces” under a log in XES – can also be handled without transformation. The event content is quite open, allowing for parts using binary encoding or attachments of various formats (e.g. MIME or SOAP attachments). However only the XML parts are queryable.

Concurrency

The notion of concurrency is as necessary to the monitoring and validation of business transactions or processes as it is to the actual programming of these processes. The concurrency of threads or activities in the latter translates into the concurrency of related event sequences in the former. A set of typical workflow patterns, several of them involving concurrency can be found in [RHAM06]. workflow language like WS-BPEL [WSBPEL07] uses an advanced set of constructs to manage concurrency and synchronization (<flow>, <links>, <joinCondition>, <sources> and <targets>).

However, the monitoring of concurrent sequences of activities does not require the same sophistication in concurrency control as the actual execution of these activities, provided the events associated with these sequences have a known correlation logic, which is generally the case in choreographed eBusiness transactions (e.g. messages relating to the same procurement action will report the same PO reference) and business process instances (e.g. a process instances initiated by an insurance claim, will generally report the claim number or originator ID across all its activities and events). Consequently XTemp has a minimal set of constructs for concurrency and synchronization: (1) the concurrent – or “non-blocking” - invocation of a scriplet, (<sart scriplet=”abc” vptsync=”false”>) for tracking overlapping sequences of related events, and (2) a synchronization operator <wait>, for joining a group of concurrent scriplets if necessary. Most other forms of synchronization simply rely on event catching <catch>.

The VP-time assigned to a scriplet invocation determines which events are accessible to the scriplet. This makes it possible to execute the same script either live or deferred over a log of events with same result, just by setting its initial VP-time to the same date/time relative to these events. Two scriplets are concurrent if their VP-time windows overlap. Concurrency is “virtual” in XTemp to the extent that the notion of “present time” (of execution) is manipulated as a data item. Even when the objective is only the analysis of past logs, concurrency actually simplifies the test scripting by allowing event processing to follow the logic of a particular sequence regardless of its context (e.g. the processing of interleaved logs of PO transactions can be done by concurrent scriplets tracking each transaction).

The XML practitioner take-away:“Core” XTemp, including virtual concurrency, is XSLT2.0-implementable for non-live executions over logs of events: this makes it very easy to verify the correctness of concurrent script over various events scenarios, before executing live. When a script has been validated over a log of events, the only modification needed before executing it live is a change in the @vptset value of its <start-with> statement in he execution context. The implementation of virtual concurrency is greatly simplified when the concurrent threads (here scriplets) do not communicate with each other. This is the case for the “core” conformance level of XTemp (no event posting). In such cases, an XSLT implementation is possible.

EXAMPLE 3: Monitoring interleaved PO transactions

The following script is monitoring PO transactions. Each PO transaction is comprised of a PO request, and a returned PO response. PO transactions are generally intertwined, as many PO requests may be sent before the first response is obtained. The monitoring of each individual PO transaction is handled by an invocation of the “get-PO-response” scriplet. These invocations are non-blocking (@vptsync="false") meaning they execute in (virtual) concurrency with the main script and with each other. The concurrency is here virtual in the sense that these invocations may actually be executed serially, provided their concurrency semantics is preserved by setting the VP-time of each invocation to the time the transaction is started (PO request occurrence). Virtual concurrency makes scripting of such test cases quite intuitive, when distinct sequences of events need be analyzed that can be intertwined in their actual occurrence.

In the script below:

These three events are ordered in time: E3 after E2 after E1. The correlation between these events is based on different content:

<script-package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" schema-version-id="0.4"
                ...
		xmlns="http://docs.oasis-open.org/tamie/xtemp/200909"
		xmlns:xs="http://www.w3.org/2001/XMLSchema"
		xmlns:myreport="http://www.abc.com/TBD/report"
		xmlns:myapp="http://www.abc.com/TBD/myapp"
		>
<scriplet name="main">
    <myreport:results>
      <loop>
        <on-start>
          <lvar name="count1" type="integer">0</lvar>
          <lvar name="successcnt" type="integer">0</lvar>
          <lvar name="elapsedtotal" type="dayTimeDuration">PT0H</lvar>
        </on-start>

        <var name="po-event">
          <catch>
            <match><condition>xtemp:content/myapp:PurchaseOrder</condition></match>
          </catch>
        </var>

        <until expr="not($po-event/xtemp:event)"/>

        <var name="poresponse">
          <start scriplet="get-PO-response" vptsync="false">
            <with-param name="ref" expr="$po-event/xtemp:event/xtemp:content/myapp:PurchaseOrder/myapp:poref"/>
            <with-param name="time0" expr="$po-event/xtemp:event/@timestamp"/>
          </start>
        </var>
	
        <eval expr="$poresponse/myreport:error"/>
        <eval expr="$poresponse/myreport:messg"/>

        <lvar-next name="count1" expr="$count1 + 1 "/>
        <lvar-next name="elapsedtotal" expr="if ($poresponse//myreport:elapsed ) then 
          $elapsedtotal + xs:dayTimeDuration($poresponse//myreport:elapsed ) else $elapsedtotal "/>
        <lvar-next name="successcnt" expr="if ($poresponse//myreport:elapsed ) then 
          $successcnt + 1 else $successcnt "/>
        <on-final>
          <myreport:summary msgtype="{concat('TOTAL PO selected: ', $count1)}"/>
          <myreport:summary msgtype="{concat('TOTAL successful transactions: ', $successcnt)}"/>
          <myreport:summary msgtype="{concat('TOTAL successful elapsed time: ', $elapsedtotal)}"/>
	  <if expr="$successcnt gt 0">
	    <myreport:summary>AVERAGE RESPONSE TIME: 
              <eval expr="minutes-from-duration($elapsedtotal) div $successcnt"/> MINUTES, 
              <eval expr="seconds-from-duration($elapsedtotal) div $successcnt"/> SECONDS.
            </myreport:summary>
	  </if>
        </on-final>

      </loop>
    </myreport:results>
  </scriplet>

  <scriplet name="get-PO-response">
    <param name="ref" type="string"/>
    <param name="time0" type="dateTime"/>

    <var name="confirm-event">
      <catch tryfor="PT600S">
        <match>
          <condition>xtemp:content/myapp:confirm[myapp:poref = $ref]</condition>
        </match>
      </catch>
    </var>
    <if expr="not($confirm-event/xtemp:event)">
      <myreport:error>No confirm event for <eval expr="$ref"/> 
        received within 10 minutes of the PO</myreport:error>
      <exit/>
    </if>

    <var name="resp-time" type="dateTime" expr="$confirm-event/xtemp:event/@timestamp"/>
    <myreport:messg>Confirmation received for PO <eval expr="$ref"/>
      <eval expr="concat('-SUCCESS: message:', $ref, 
		  ' Response received within:', 
		  minutes-from-duration($resp-time - $time0), ' MIN ',
		  seconds-from-duration($resp-time - $time0), ' SEC'
		  )"/>
      <myreport:elapsed><eval expr="$resp-time - $time0"/></myreport:elapsed>
    </myreport:messg>
  </scriplet>
</script-package>
	

The XML monitoring report produced by this script over a sample event log is:

<myreport:results xmlns:translator="translator"
           xmlns:xtemp="http://docs.oasis-open.org/tamie/xtemp/200909"
             xmlns:myreport="http://www.abc.com/TBD/report"
          ...
             xmlns="http://docs.oasis-open.org/tamie/xtemp/200909">
   <myreport:error>No confirm event for Name1 received within 10 minutes of the PO
     </myreport:error>
   <myreport:messg>Confirmation received for PO Name2 -SUCCESS: message:Name2 
     Response received within:5 MIN 40 SEC<myreport:elapsed>PT5M40S</myreport:elapsed>
   </myreport:messg>
   <myreport:error>No confirm event for Name3 received within 10 minutes of the PO</myreport:error>
   <myreport:messg>Confirmation received for PO Name4 -SUCCESS: message:Name4 
     Response received within:6 MIN 0 SEC<myreport:elapsed>PT6M</myreport:elapsed>
   </myreport:messg>

   <myreport:summary msgtype="TOTAL PO selected: 4"/>
   <myreport:summary msgtype="TOTAL successful transactions: 2"/>
   <myreport:summary msgtype="TOTAL successful elapsed time: PT11M40S"/>

   <myreport:summary>AVERAGE RESPONSE TIME: 5.5 MINUTES, 20 SECONDS.</myreport:summary>

</myreport:results>
	

Generating Test Reports

The primary output of a script execution is called an X-effect (for "XML side-effect"). The X-effect is a text output, which is generally well-formed serialized XML but may also just be plain text. The notion of X-effect is a generalization of the output produced by XML script languages such as XSLT, that can be a combination of in-line XML fragments with external namespaces and of computed values or fragments.

The X-effect of a script can be recursively defined as follows:

Two XTemp features could be seen as challenging for the production of X-effects, but ended up with intuitive X-effect semantics:

Exits: The execution of an <xtemp:exit> statements will simply cancel the production of X-effects of its following siblings (as these are not executed), meaning the X-effect of the embedding statements is preserved. The rule also applies to the following siblings of any conditional statement that executes <xtemp:exit> at any level inside.

Concurrent executions:The rule for X-effect generation of a scriplet invoking another scriplet, is exactly the same regardless whether this invocation was blocking or non-blocking (i.e. serial or concurrent): The combined X-effect of scriplets where one is started by the other, is following these rules: When scriplet S1 "starts" (either blocking or non-blocking ) scriplet S2, the X-effect of S1 is obtained by : (1) capturing the X-effect of S2 at the end of S2 execution, (2) substituting the <starts scriplet="S2" vptsync="..." /> statement in S1 with the X-effect of S2. (3) generating the final X-effect of S1 at the end of its execution, after substitution in (2). This rule applies recursively to all scriplet invocation trees, possibly combining blocking and non-blocking invocations. The rationale for this, is that the tree-like structure of an XML report must generally match the hierarchy and order of scriplets *invocations* regardless of the actual course of their execution which may overlap or be concurrent in various ways. For example, the report produced by the previous script example is naturally listing the results for each PO transaction in the order these PO were initiated – as a sequence of transaction reports, as a human reader would expect -, regardless of how these transactions executions were intertwined, e.g. even if the first PO transaction is also the last one to complete.

The XML practitioner take-away: The production of XML test reports is most intuitive and readable when combining in-line XML fragments of foreign namespaces and computed fragments or values, as done in XSLT. Concurrent executions in XTemp, as well as breaking statements such as <exit> are not an obstacle to the use of these report generation techniques familiar to XSLT developers.

XTemp and XSLT

XSLT is computationally sufficient to process events in a 'deferred' mode (static analysis), assuming a representation of event boards as static XML files. However, when considering whether XSLT alone should be used - e.g. by defining appropriate design patterns for event handling and timing - the issue of usability and developer-friendliness arise. The declarative, functional design of XSLT based on scripts driven by template matching does not make it an intuitive language for workflow practitioners or more generally for analysts and developers familiar with the scripting of business processes designed along a state machine model with clearly defined state transitions for underlying data. It is the experience of the authors that it is critical to provide constructs familiar to target users and at the right level of abstraction, for a language to be well accepted. Consequently XTemp defines or imports concepts familiar to business process and workflow developers such as event catching, concurrent paths control and synchronization, timing of state transitions, exiting, mutable variables. The experience of the authors in analyzing and testing event traces is also that a balance must be reached between pattern-driven language with strong functional design such as XSLT and workflow languages like WS-BPEL.

On the other hand, XSLT and XPath have reached a level of maturity and expressiveness that cannot be ignored, and XTemp is leveraging both to a large extent. XTemp1.0 is using XPath as expression language and relies extensively on its functions and operators library, in particular for its time and duration processing (e.g. XPath2.0 library). Shouldn't then XTemp be defined as an XSLT extension? Although such an approach is not ruled out in future versions, a couple of reasons weight in favor of keeping its syntax separate for now:

In spite of these open questions the XTemp design is still influenced by XSLT features - i.e. it matches feature-wise a subset of XSLT and its new features (variable mutability, event catching and time handling) are designed to be XSLT-implementable, even if not in an optimal way, at least for deferred executions - or analysis of static event boards. Implementability using XSLT - even for a restricted ('deferred') usage and for a subset of XTemp features called 'core' XTemp - is an important aspect of usability of the language.

Among the features of XTemp beyond the 'core' subset (as defined by the 'core' conformance level in [XTEMP10]) and considered challenging or impossible using conventional XSLT engines, are:

Another approach in analyzing and testing the trace of business processes, is to rely on a simpler and more declarative 'rule' language (or 'pattern' language) usually implemented in XSLT, such as Schematron or Tamelizer (the latter being a recent implementation - see: http://code.google.com/p/tamelizer/ - of a test assertion language proposed in [TAG09] by one of the authors.) Such declarative languages in turn rely heavily on the expressive and processing power of XPath for the testing logic. But declarative test assertions or patterns are of little help when expressing complex test logic spanning sequences of events: although Xpath2.0 and above has a remarkable ability for expressing such correlation and validation logic in a single predicate expression, the readability and maintenability of such tests is poor. For example, the test suites for WS-I [WSI10] were based on the above test assertion design and related analyzer tool. In these test suites XPath2.0 alone proved indeed able to handle complex choreographies of timestamped inputs such as Web services requests and responses, combined with related low-level messages such as resends or receipts and various hand-shaking procedures. But the resulting large XPath expressions are hardly readable and difficult to evolve or to update, requiring a level of skills and efforts in XPath programming beyond what most test engineers are ready or willing to exercise. The ability of above languages to chain test assertions or to compose rules, does not really help to break down this kind of complexity. It became clear that a more powerful yet more intuitive combination of logical expressions and procedural flow definition such as in XTemp, was needed.

WS-BPEL and Workflow XML vocabularies

Workflow languages such as WS-BPEL [WSBPEL07] and others XML vocabularies or notations (XPDL, BPMN) are intended to describe and drive business processes, more than to analyze and verify their behavior. To this extent, even if they share several seemingly equivalent algorithmic and synchronization constructs with XTemp, their purpose is rather complementary. The notion of event is used in WS-BPEL mostly as a dynamic input to business process instances, and not as an input for a more general analysis that may span several processes. The notion of correlation set is used for helping the selection of the next event that belongs to the same process, which is expected to depend on a predefined set of event attributes that define some event equivalence class. To this end, correlation sets are sufficient and efficient but unable to express more general event patterns needed in analysis and live monitoring. The concept of event handler is intended in WS-BPEL as a device for helping drive process instances, typically as these events occur. It does not assume the capability of processing past events. Finally, the focus on analysis makes it possible for XTemp to only use a minimal set of constructs, compared with WS-BPEL.

XProc and Pipelining

Although XSLT, XProc and XTemp share the same general purpose of advanced processing of XML inputs, and share several similar algorithmic constructs, it is interesting to stress the different nature of the entities behind these XML inputs and of their processing. The target unit for processing in XProc is a “document”, while it is an event in XTemp. Clearly, this distinction is not as much about content (a document can be represented as an event and vice-versa) as it is about the general nature of the processing of these items – as well as how these items relate to each other. In short, documents are to be validated, transformed, compared, merged. Events have to be matched, correlated, waited for or queried.

A combination of documents that has to be part of the same processing flow is statically known in general (e.g. a business document + its possible includes + its XML schema + a set of codelists + some semantic rules). Their role is well established as these documents will be assigned specific ports in an XProc pipeline. In comparison, the set of events of interest for processing as a unit is generally unknown beforehand, determined dynamically by a tentative match with a pattern definition that may or may not succeed. The document itself has a business value, and the value-add from Xproc is in its (efficient) validation, transform, or assembly. Events generally have value not through individual processing, but in relation with each other. Consequently, the ability to correlate efficiently over a potentially large event repository is key in XTemp, while advanced, multistep processing and related pipeline optimization is not its goal.

Similarly, although XSLT is computationally sufficient by itself to drive a chain of document processing or to analyze a log of events, its constructs do not support these paradigms in a way that is specific enough for practitioners to feel comfortable with the language, or for implementors to use appropriate modules or architectures.

Testbeds using XML vocabularies for Test Suite syntax

XML vocabularies have been used in B2B testbeds for test case and test suite representation. Two of the authors have been involved in an earlier, very limited version of XTemp [ETSL07] that did not have the same algorithmic capabilities nor leveraged XPath to the same extent. Implementations proved sufficient for the driving and verification of many B2B test cases, but lacked the ability to generate advanced reports that require data consolidation over several B2B exchanges. Other testbeds have also used XML vocabularies for test suites, that were developed at NIST [WSTB2B06] and also in [TESTHL709]. These testbeds represent the state of the art in XML-based test suites. However they do not leverage fully the notions of event and test execution concurrency as means to a powerful analysis of B2B logs. Their focus is more on driving test cases than on test analysis.