How to cite this paper
Mercier, Benoit. “Including XSLT stylesheets testing in continuous integration process.” Presented at Balisage: The Markup Conference 2011, Montréal, Canada, August 2 - 5, 2011. In Proceedings of Balisage: The Markup Conference 2011. Balisage Series on Markup Technologies, vol. 7 (2011). https://doi.org/10.4242/BalisageVol7.Mercier01.
Balisage: The Markup Conference 2011
August 2 - 5, 2011
Balisage Paper: Including XSLT stylesheets testing in continuous integration process
Benoit Mercier
Analyst
Faculté des lettres et sciences humaines, Université de Sherbrooke
Benoit Mercier has been working as researcher and analyst at the Franqus research
group of the University of Sherbrooke (Quebec, Canada) since 2006 where he collaborates
to the development of a new North American French dictionary. His main interests include
computer linguistic, Free Software development and technology watch. His is also a
European Commission official since 2000. He started and animated the Commission IT
Network (CITnet), an internal collaboration platform for IT specialists (almost 2000
individuals) and is the author of the 2003-2006 version of the strategy for internal
use of Open Source Software at the European Commission.
Copyright © 2011 Benoit Mercier
Abstract
XSLT stylesheets are just like any other programming language code units: they
need to be written, debugged, refactored and tested. In these days of « Agility »,
refactoring and continuous integration play major roles in development process. How
to assert that XSLT stylesheets refactoring are still producing correct outputs? How
to early detect defects? This paper proposes a practical approach to include XSLT
stylesheets testing in continuous integration process, based on XSpec, a Behavior
Driven Development (BDD) framework for XSLT, and on Java helpers
provided by Jxsl, a Java
XSL code library.
Table of Contents
- Problem
- Solution
-
- XSLT test framework
- Java unit testing wrapper
- Build tool and continuous integration server
- Conclusion
- Acknowledgments
Problem
Writing XSLT stylesheets is similar to writing code in any kind of programming
language. The aim is to build a working piece of code that will produce predictable
results to meet functional requirements. Software development is evolving at a fast
pace and more and more new methodologies are emerging in order to guide development
process. One of the key aspects shared by all these methodologies is testing, to
guarantee software behaviour predictability and consistency. To be really efficient
and
useful, testing should be made easy for developers and tests scenario executions should
be triggered automatically as soon as code has been modified. This is nowadays part
of a
common best practice called Continuous Integration (CI). But XSLT stylesheets are
in a
poor relation in this context. There are few unit testing frameworks available for
XSLT
and none of them are easily embeddable in CI processes and tools.
This paper tries to demonstrate that XSLT stylesheet testing can be made simple by
using the XSpec
Behavior
Driven Development (BDD) framework for XSLT through the use of Jxsl Java wrapper library and can
be easily integrated into existing or new CI environment. It is an awareness-raising
document to help XSLT stylesheet developers to change their outlooks so as to foster
the
adoption of testing as a new valuable tool at their disposal.
Solution
The proposed solution to achieve continuous integration of XSLT stylesheets adds three
possibly new components to the usual toolkit (IDE, version control system, etc.) used
by
XSLT developers:
The following sections describe the selected tools and their respective roles in the
process.
XSLT test framework
This is the core of the solution. Writing tests should be easy, intuitive and
frictionless. XSpec framework meets these conditions[xspec-01]. BDD approach, tool quality,
authors and recent project activities have lead to its selection from among various
other projects like Juxy, XSLTUnit or UTF-X for instance.
The general idea is to write XSpec documents to describe stylesheet behaviours.
XSpec documents are XML documents which adhere to the XSpec
RELAX NG schema. XSpec scenarios are straightforward to write for XSLT
developers. Stylesheet templates and functions can be tested against contextual data
(XML documents, fragments or mock object) by creating test scenarios. Here is a
sample XSpec file.
<?xml version="1.0" encoding="UTF-8"?>
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec" stylesheet="toHtmlWithIds.xsl">
<x:scenario label="When processing a list of books">
<!-- apply template rules to this element -->
<x:context href="xsltestengine-data/books-a.xml/>
<!-- check the result -->
<x:expect label="There must be 12 button to books" test="count(//button) = 12"/>
</x:scenario>
</x:description>
The precise way to write XSpec document is described in the official documentation
[xspec-02].
Systematically writing XSpec test scenarios for each XSLT stylesheets produced is
a first step towards improving bug detection, guaranteeing compliance to
specifications, business and technical requirements, safer refactoring, etc. But
XSpec documents alone are not sufficient. Running tests is still under the sole
responsibility and good will of the developer. Tests should be run on a regular
basis to allow early error detection. Such systems exist and are widely used for
other programming languages. How could XSpec files be integrated in those existing
solutions?
Java unit testing wrapper
In order to be able to benefit from other languages testing tools availability,
XSpec test executions should be wrapped up in one of these languages. This is one
of
the goals of the Jxsl project [jxsl-01]: to offer Java wrapper objects for XSpec test
configuration and executions.
Jxsl provides method to execute XSpec tests directly from Java code, one at a time
(XspecTestScenarioRunner) or in batch mode
(XspecTestSuiteRunner). This is useful for
embedding test executions in custom Java code. But Jxsl also contains wrapper for
the JUnit unit testing framework (TestNG support is coming). That means that an
existing Java project can add XSpec file testing to its standard JUnit test
suites.
To enable XSLT unit testing with XSpec in an existing Java project, it is as
simple as creating a new class in the project test package that extends XspecScenarioJUnitTest and to provide a Spring bean
configuration file called xspec-context.xml. Here is a complete and functional
implementation.
package com.mycompany.test.xspec;
import com.servicelibre.jxsl.scenario.test.xspec.XspecScenarioJUnitTest;
import java.io.File;
public class XspecUnitTesting extends XspecScenarioJUnitTest {
public XspecUnitTesting(File xspecFile) {
super(xspecFile);
}
}
Jxsl offers a Maven archetype to get started quickly. This archetype is also very
useful for non Java developers. Java 1.5+ JRE and Maven 2.9+ are required. The
archetype can be generated with the following command:
mvn archetype:generate -DarchetypeGroupId=com.servicelibre \
-DarchetypeArtifactId=xspec-test \
-DarchetypeVersion=0.1.5 \
-DarchetypeCatalog=http://jxsl.googlecode.com/svn/trunk/archetypes
The only things to configure to get started is the XSpec file locations . This is
achieved by editing the src/test/resources/xspec-context.xml
file.
Tests can now be run with the following
command:
mvn test
XSpec test executions have now be integrated into some Java unit testing code in
order to benefit from all the tools available for triggering and monitoring standard
Java unit tests. The Maven project build from the archetype could also be used to
integrate XSpec testing in XML editor like Oxygen XML Editor (through External tools
configuration).
Build tool and continuous integration server
XSpec tests wrapped up in Java unit tests can now be triggered as part of a
standard Java project build process. Build tool like Maven does this automatically
by convention. Ant can be configured to run JUnit tests as follows :
<?xml version="1.0" encoding="UTF-8"?>
<project name="jxsl" default="test" basedir=".">
<target name="init">
<property name="resources" location="${basedir}/../resources"/>
</target>
<target name="test" depends="init">
<echo>basedir=${basedir}</echo>
<junit dir="${basedir}" fork="true" haltonerror="true" >
<test name="com.servicelibre.jxsl.scenario.test.xspec.XspecScenarioJUnitTest"/>
<classpath>
<pathelement path="${resources}/jxsl/jxsl-with-dependencies.jar"/>
<pathelement path="${user.home}/.m2/repository/net/sf/saxon/saxon/9.3.0.2j/saxon-9.3.0.2j.jar"/>
<path path="${basedir}"/> <!-- for xspec-context.xml Spring bean configuration file -->
</classpath>
<formatter type="brief" usefile="false"/>
</junit>
</target>
</project>
Today, not having a CI server in place when doing software development is almost
like not using a version control system: a risky bet. CI server ensures, at least,
that code are regularly built and tested successfully in a clean environment
(understand « not on the developer computer ») . There are several popular CI
servers: Hudson/Jenkins, Continuum, Bamboo, TeamCity, etc.
XSpec tests should be run regularly by a CI server.
In order to catch errors even more early, it is suggested to trigger test
executions on commit in the version control system. This is usually done via
server-side hook scripts. Resources on hook scripts for common version control
systems:
Here is a sample Subversion hook script that triggers a project rebuild by
Hudson
#!/bin/sh
# POST-COMMIT HOOK
USER=hudson
PASSWORD=*****
HUDSON_SERVER=hostname:8081
HUDSON_COMMAND=polling
HTTP_PREFIX=http://${USER}:${PASSWORD}@${HUDSON_SERVER}
wget -b ${HTTP_PREFIX}/job/hudson_project_name/${HUDSON_COMMAND} > /dev/null
Conclusion
With the proposed solution, XSLT developers, even without any knowledge of Java, are
now able to easily create a Maven project from an archetype, configure it to run their
own XSpec files and trigger test executions by a CI server at each commit in their
version control system.
Acknowledgments
This project would not have been possible without the support of the Franqus Research
Group of the Faculté des lettres et sciences humaines, Université de Sherbrooke. The
author wishes to express his gratitude to colleague and friend, Dominic Marcotte,
who
offered invaluable assistance and support.