About this Paper

The sheer number of new features added to the 4.0 specifications[1] makes it very difficult to provide a concise summary of what's new. Even in the release notes[2] for Saxon 13, which implements many of the new features, we resorted to the headline summary "More than 175 features from the draft 4.0 specifications are now available...." There are various ways of tracking the detailed changes: the specification itself contains a "changes" summary below the section heading of any section that contains substantive changes, and these link to the GitHub Pull Requests[3] that contain the detailed text of each change. Test cases in the extensive test suite[4][5] (also available on GitHub) are also linked to pull requests, and pull requests that have been implemented in a released Saxon version include a GitHub label "In Saxon 12" or "In Saxon 13".

As a taster, I published a list of 101 Fun things to do with QT4[6]: none of them described in any detail, but a list intended for easy reading where, if particular features whet your appetite, you can go to the specs and investigate further.

So I won't make any attempt in this paper to provide a comprehensive list of new features. Instead, I will take a step back, and try to discern some overall themes.

What is the status of the work? It's six years since I first proposed the QT4 initiative [Kay 2020], and four years since the QT4 community group started work to flesh out a definition of the 4.0 versions of XSLT, XPath, and XQuery. The work is ongoing and subject to revision and change as the work proceeds.

At a process level, there is still some work to do to decide under what auspices the work should be published. W3C no longer has a formal XML Activity, and there are no official working groups chartered to take the XML standards forward. QT4 is not alone in making technical progress through the informal mechanism of W3C Community Groups[7], but these do not have the authority to publish official Recommendations. As the work nears completion, we can expect to see the politics get resolved, but meanwhile the work is presented to the world to accept or reject based on its technical merits.

Process is important, but the significant parts of the process to date have been those that facilitated the technical collaboration to produce a high quality specification, rather than the process for getting an official stamp of approval on the final publication (including a formal consultation process). Because process is important, I'll share my thoughts on the subject.

The current state of the QT4 standards

We can divide this into three parts.

  • What new features are there in the spec that appear to be stable and complete and require no further work?

  • What features in the spec are known to be incomplete and should be considered work in progress?

  • What requirements remain to be addressed?

I'll consider these three questions in the following subsections of the paper.

Completed work

It's never over till the fat lady sings, as they say, so we can never say definitively that a feature already added to the spec is finished and stable and won't change before final publication. But there are some things we can be 95% confident about. Or at any rate, we can be confident that any further changes in these areas will be fairly superficial.

In this category I would include the following:

  • Most of the 92 new functions added to the core function library[8], and the 154 extensions to the behavior of existing functions.

    Many of the new functions are very simple, but hopefully also very practical.

    Some are pure convenience functions, like fn:characters which splits a string into single characters, or fn:is-NaN which tests whether a value is the special floating-point value NaN, or fn:foot which returns the last item in a sequence.

    Many of the new functions are higher-order, producing effects that would otherwise have required custom syntax: some of my favorites are fn:highest and fn:lowest which return the items in a sequence that have the highest or lowest values for some computed property; fn:take-while which returns all the items in a sequence up until the first that satisfies some condition; and fn:all-equal and fn:all-different which test respectively whether all items in a sequence are equal, or whether they are all distinct. Also fn:transitive-closure which does what the name suggests.

    Other new functions incorporate powerful new capabilities to process external data formats. In this category we have fn:parse-html and fn:parse-csv to process HTML5 and CSV input data respectively, and fn:invisible-xml to process arbitrary input conforming to a user-supplied grammar. Another big one is fn:element-to-map which does XML-to-JSON conversion according to the design outlined in [Kay 2023]. More modestly, fn:parse-uri splits a URI into its parts, and fn:build-uri puts them back together again.

    Many existing functions have acquired new options. fn:deep-equal, for example, has options to control exactly how comparisons are performed, such as whether whitespace and namespace prefixes are to be considered significant. The fn:doc and fn:document function have options to enable DTD and XSD validation and XInclude processing. The fn:round function has options to control rounding mode.

  • One of the benefits of higher-order functions is that many powerful capabilities can be added to the language without new syntax. But I think some of the XPath syntax extensions[9] will also prove popular. How did we ever manage without the otherwise operator? - the expression @discount otherwise 0 return the value of @discount if the attribute exists, or zero otherwise. Extensions to for expressions in XPath (or the for clause of FLWOR expressions in XQuery) to process maps and arrays will probably find ready acceptance. And I think the abbreviated syntax for simple inline functions will also have a big impact: writing a callback as fn{@name} rather than function($e as element(*)) { $e/@name } doesn't just save keystrokes, it makes a substantive difference to the aesthetics and therefore the usability of higher order functions.

  • On the XSLT front[10], there are also a few simple features that look finished and stable. The xsl:if instruction now has then and else attributes, while xsl:when and xsl:otherwise have a select attribute, and there is a new xsl:switch instruction for a multiway conditional. The ability to write all the template rules for a mode as children of the xsl:mode element provides a way to make code more readable and therefore easier to debug.

  • Generally we have tried to do as much as possible without significant changes to the data model[11] or type system, because such changes are always more disruptive. But a few changes in that area stand out as being particularly successful:

    • Choice types, for example, (xs:date | xs:time | xs:dateTime), or (element(E) | document(element(E))).

    • Enumeration types, for example, enum("red", "green", "blue").

    • Changes to the function conversion rules, now renamed coercion rules, which allow downcasting: you can specify the required type of a function parameter as, say xs:positiveInteger, and the caller can supply the value 42 without explicitly casting it to that type: the cast will happen automatically. This makes the use of derived types such as xs:positiveInteger much more attractive.

    • Maps now retain order. The immediate benefit is that when you perform a transformation on JSON input, and serialize the result, the output is now readable rather than being garbled.

Work in Progress

There are a number of areas in the specifications where a great deal of progress has been made, but a number of issues remain open. These tend to be related and intertwined, so progress in one area can involve changes in another.

At Balisage 2025 I reported on the introduction of JNodes [Kay 2025], and most of the ideas presented in that paper found their way into the draft specification, and have proved robust. But:

  • There are some loose ends regarding the handling of arrays whose members are XDM sequences rather than single items; and similarly, maps whose entries are general XDM sequences. These constructs never arise from parsing JSON, but they can arise in the general case of a constructed tree, and we need to work out the detail of what to do with them.

  • The proposed syntax $a/get("date of birth") (also available with other axes, for example, as $a/descendant::get("date of birth")) met with some resistance. The current proposal, which still needs a little work to finalize, is to substitute the syntax $a/"date of birth" or $a/descendant::{"date of birth"). In both cases, in principle, the string literal "date of birth" can be replaced with an expression, but this potentially leads to conflicts with the semantics of expressions such as $a/name() or $a/jvalue().

  • While the use of JNodes in path expressions for selection within JSON-derived trees is now quite well defined, there are still open questions[12] about transformation of such trees using the XSLT recursive-descent processing model with template rules. What should the syntax be for match patterns that match JNodes? What should their default priority be? What should the built-in template rules do when there is no explicit match? These areas are all still being explored.

Record types were introduced in the very first proposal for 4.0: the idea is that the type record(partNr as xs:string, price as xs:decimal) should be descriptive of a map containing two entries, an entry with key "partNr" whose value is a string, and an entry with key "price" whose value is a decimal number. This correlates well with a common way that maps are used, and many of the new functions and operators have found record types convenient to use in defining their argument types and result types. But as with JNodes, record types have been subject to some changes as the spec has progressed, and we're probably not done yet. Some of the areas of change have been:

  • We have abandoned the flag that defined a record type as being extensible (allowing it to contain fields other than those listed). This went against the aim of strict typing for record types, allowing an expression such as $product?part-nr (with a misspelled field name) to be flagged as a static error.

  • We've also dropped the ability to define some of the fields as being optional. We found there was too much scope for confusion in that absent data could be represented either by a field that was absent, or by a field that was present but empty. In the current design, absent data in a record is always indicated by a field whose value is an empty sequence.

  • As originally defined, a record type was simply a predicate applied to a map. We have now introduced the idea that a map should have a type annotation identifying it explicitly as an instance of a specific map type. But I don't think we have yet worked through all the implications of this change, for example, the precise rules as to whether two record types are "the same type".

  • As a replacement for the ability to define record types as extensible, there's a suggestion that we might define one record type to be an extension of another.

  • We have syntax in both XQuery and XSLT to declare named record types. One of the advantages of making them named is that record types can become recursive: for example, a LISP enthusiast might define the record type my:list as record(car as item()*, cdr as my:list?): a two-entry map containing an arbitrary data value called cdr, and an optional [pointer to a] my:list called cdr. I don't think we've yet tested the implementability and usability of recursive record types in anger, and there might be changes once we do.

  • We've defined that named record types should have an associated constructor function to create instances of the named record type. But we've also found use cases where no constructor function is wanted (in particular, where the record type is designed explicitly to be the result type of a function such as fn:parse-uri) and we've also found cases where a customized constructor function (or set of constructor functions) is preferred over one that is system-generated.

Records have proved their worth and will no doubt feature in the final spec, but there are still rough edges to be smoothed out.

Outstanding requirements

As well as features that appear to be done and dusted, and other features that are done but still need some dusting, we have a wish-list of bright ideas[13] that someone would like included in the spec. In some cases there is consensus that these features would be nice to have; in other cases their value is disputed. What characterizes them all, however, is that the spec could be published without them: they are not essential to declaring victory.

Some of these ideas come from within the Community Group, some come from outside. Some of them are well formulated proposals, others are sketchy ideas. The problem is, they are opened faster than we can close them.

Every working group hits the problem that at some stage it wants to declare itself finished, but it still has this pool of good ideas for making the spec better. Essentially there's a social process within the working group when the consensus starts to form that we're in the end game and must politely decline to do any further work. Of course, there's still a tendency to accept ideas if someone steps up and does all the detailed work to define the precise syntax and semantics and to write test cases. But even then, working groups eventually lose patience and declare that the proposal is out of time.

But until then, there are still some intriguing ideas on the table. There's a big proposal on generators[14]: functions whose repeated application produces an infinite sequence. Dimitre Novatchev is planning to present that proposal at this conference. Other proposals are more limited in scope: a proposal to allow comment lines in CSV files, for example, or to add named capture groups in regular expressions, or to add a function that converts maps to element nodes, or to do "deep update" of a tree derived from JSON processing. The Community Group will only be able to declare itself finished when it has made a decision (one way or the other) on all these good ideas.

Standardization in the XML community

The XML community's attitude to standards is driven by the nature of XML itself. As a standard for transmission of data between heterogenous systems, and even more so as a standard for long-term storage of information of lasting value, XML instrinsically needs to be very rigorously standardised, and moreover, changes to the standard over time are generally unwelcome, as proven by the failure of the 1.1 version: the cost to the community of adopting a change exceeds the benefit. The same observations can be made for JSON, and it is notable that for both XML and JSON, only very small changes have been attempted once they had become established in widespread use. Even fixing widely-acknowledged bugs (such as the fact that JSON allows unpaired Unicode surrogates, or the fact that XML fails to say exactly what constitutes a valid namespace URI) has proved remarkably difficult.

This community attitude to standardisation naturally extends to closely associated standards such as XML Schema. The need for standards in areas such as transformation and query languages is perhaps not as strong; there is no intrinsic reason why multiple programming languages and APIs (and successive versions thereof), should not thrive just as happens in the field of general-purpose programming languages. What's different here is the architecture of the web: if stylesheets are to be executed in the browser, then the stylesheet language needs to be supported by every browser, and until recently, the browser environment has been a barrier to innovation because the platform is tightly controlled by a cabal of browser vendors. The standardisation culture for the XSLT environment is thus conditioned partly by the original motivation of client-side transformation, which demands rigid standards, and partly by the area where it has seen most success, as a server-side domain-specific transformation language, where there is more room for variety and innovation: a culture that has enabled newer versions to thrive in a way that newer versions of XML and JSON could not.

Lessons from history

QT 4.0 is, as the number suggests, the fourth phase of standardisation in this area. The 1.0 standards were produced very quickly, and proved highly successful, though they had many technical flaws. Each subsequent phase has taken longer: seven years to version 2.0, a decade to version 3.0, perhaps even longer (who knows) to version 4.0. On many measures the standards have improved in quality: they are more precise, less ambiguous, better supported by test suites, there are certainly fewer gaps in functionality, and fewer usability quirks. But some of the new features that were introduced have not achieved widespread adoption[15]. Can we look at the features that were successful, and those that weren't, and spot any trends?

My personal attitude is that the success of a standard, and of new features in a standard, is more related to the cost of adoption than to the benefit of adoption. The most significant innovation in XSLT 2.0 was schema awareness, and twenty years later it seems that only a small minority of the user community is using the feature. This is not because there is no benefit: our experience suggests that when developing complex XSLT-based applications, schema awareness gives many productivity benefits through easier debugging. But there is a significant up-front adoption cost. Writing the first version of your code takes longer, even if you get to a complete debugged application more quickly. Similar observations apply to some of the big ideas in 3.0, such as streaming and packages, both of which deliver significant benefits that only a small number of enlightened users are taking advantage of.

By contrast, the things that have been most successful have been those that could be adopted incrementally without any major change. In XSLT and XPath 2.0, grouping, regular expressions, XSLT functions, date and time manipulation, conditional expressions. In 3.0, maps, xsl:iterate, accumulators. Higher-order functions occupy a middle ground: the cost of adoption here is primarily the challenge of learning a new concept in computer science and software engineering, and adoption has steadily increased as the same basic idea has become familiar in other languages such as Java, C#, and Javascript.

I think that the vast majority of changes in QT4 are incremental in this sense: once users discover that the feature exists and is available in their chosen products, they will be quick to take advantage of it.

The standardization process

Who are the stakeholders in the standards process?

I will identify four: the users, the implementors, the funders, and the standards organizations themselves. What we end up delivering depends on how these four groups of stakeholders play the game, and that in turn depends on what they are trying to achieve.

What I'm going to say here draws not just on experience with QT4, but on other experience within W3C and before that with other standardization projects: Codasyl, ANSI, BSI, ISO, ECMA, in some of which I was a direct participant and in others merely an interested observer. And this over a timespan of fifty years.

The mechanics of the standardization process have of course changed immeasurably over that timeframe. In the Codasyl days I would type a proposal on a typewriter, make 40 copies, and address 40 envelopes, making sure they arrived three weeks before the next meeting, which was always somewhere in the US and lasted a week. Compare that to the process of creating a pull request on GitHub, and getting feedback an hour later. But although the mechanism may have changed, the factors that make some proposals successful and others unsuccessful have not.

On most of the groups I have worked with, users have acted primarily as reviewers rather than initiators. A genuine user (by which I mean someone who actually does hands-on development and knows the specs well) is a great asset, even if review and verification is the only contribution they make. Unfortunately some user organisations instead send a supernumerary, someone who is no longer actively engaged and whose technical knowledge is rusty; these people can be time-wasters. The QT4 activity has been mercifully free of time-wasters, though we do of course get the occasional contribution from someone outside the core group who is unaware that the feature they want is already available.

By "implementors", I mean those participants who are actively involved in creating implementations. I don't mean representatives of the companies who market products, I mean the designers and developers who actually cut the code. Very often there are only two or three genuine developers taking part. That's not necessarily a bad thing: the original XQuery 1.0 work was hampered by having perhaps eight or nine people in this role, many of whom could have done a good job as chief designer of the language, but all of them having different perspectives on what kind of language they wanted to create. It tends to be the developers on the group who come up with most of the ideas. Ideally they not only develop implementations of the standard they are working on, but also have projects in which they develop applications using the technology, so they can put themselves in the mindset of users. They typically also have their own source of feedback from their own user community.

In my experience the developers on the group, although they work for companies that are competing with each other, often get on very well with each other: they love sharing ideas and often build on each others' ideas to create something that none of them would have come up with individually. This collaborative competition is highly beneficial to improving the state of the technology for everyone. There can be tensions, however, if very capable people share different visions of the end product.

All groups can suffer from having participants who are poor team players. These people may be very able technically, but they can get so passionate about their good ideas that they lose any ability to see other points of view and make compromises. They can also lose sight of the fact that selling their ideas is essentially a human process that depends on understanding where other people are coming from. The chair of one standards group, who in his day job was an industrial psychologist, once took me aside and told me that a colleague of mine had (in my absence) presented a proposal in such an antagonistic manner that the group rejected it out of hand, even though the proposal actually had considerable technical merit.

The funders — the people who pay the salaries and hotel bills of the participants — have their own agenda, but in my experience, they are often rather vague about what they actually want to achieve. It's hard to convince them that they are in a joint venture with their competitors from which everyone gains an advantage. They may well be under the misapprehension that by taking part, they will be able to get features into the spec that benefit them and harm their competitors: an argument which, in my view, is nearly always deluded. The real business case is that the activity grows the size of the market and they get their fair share of that growth.

Too much input from the funders can have detrimental effects. When vendors start pushing the standard in a particular direction, the pressure is often unrelated to any genuine understanding of user requirements. Whether the pressure is from managers in the vendor organization, or their marketing people, or the IT directors investing good money in the vendor's products, the chances are that the real needs of hands-on developers using the technology have got lost somewhere along the way. This can easily lead to complex features being added that no-one ends up using, while simple practical improvements are overlooked.

There's a contradiction I've observed in all the groups I've worked with. The fine detail of a new feature gets an enormous amount of detailed attention, and ends up being very well specified and documented (far better, in my experience, than the specification of any proprietary software product). Meanwhile, the requirement behind the feature often receives little scrutiny. If someone claims that their users need the feature, it's very hard to challenge that statement; standards groups gather their requirements from a small and self-selecting sample of users. Certainly, I've never seen any serious attempt to quantify the cost benefit of a feature, or to apply any rational process to prioritization of different features.

Finally, the standards organizations: W3C, ECMA, ITU, IETF, WhatWG, or whatever. What is their role and motivation? Almost invariably, these groups start as a collaboration between vendors, often vendors whose common interest is to reduce the dominance of an existing monopoly supplier. If they are successful at all, the dominant supplier later senses which way the wind is blowing and joins the group. Very often the group is formed because established venues for doing the work have become fossilized: they have built up sufficiently heavyweight decision-making processes that the active parties decide they are better off starting something new. Whether the rest of the community likes it or not, it is implementors who drive the technology forward. User input is useful, but the real costs are borne by implementors, and they are only going to keep implementing a standard if it's in their interests to do so. There was a stage in the 1980s when official "de jure" standards bodies became totally disconnected from implementation activities, and it led to the creation of numerous standards that weren't worth the paper they were written on. As the internet sprang to life, it became clear that implementors had the upper hand, and that standards were often written as a post-hoc exercise once the defining products had already become established. That's true regardless whether the products in question were open source (like the early email software, or like browsers) or whether they were proprietary (like Microsoft Office).

What does all this mean for QT4? We've been fortunate. There have been no misguided inputs from funders disconnected from hands-on users, there's a general feeling that everyone is pulling in the right direction, and we benefit from the reviews and comments of a wider user community that understands the technology well and knows its limitations. We have developed our own very effective and economical tooling based on GitHub, XML, XSLT, Gradle, and the proprietary DeltaXignia product (formerly DeltaXml). The absence of a sponsoring standards organisation (unless you count the very hands-off W3C, which makes no attempt to control the direction) hasn't hampered us so far, though it might be useful to re-establish a branding relationship in order to build credibility as we come to publication stage. But we might find that at that stage, politics creeps in.

We do have a problem knowing when to stop. We're all perfectionists who want to get it right, and we tend to prioritize quality over completion dates.

One of the obstacles W3C put in our way during the development of the 3.0/3.1 versions of the standards was the requirement to demonstrate the existence of three independent implementations. We scraped through that requirement on XSLT 3.0: we had two implementations actively developed by working group members (Saxon and Exselt, the latter having since been withdrawn from the market), and there was a third implementation, Altova Raptor, which followed the standard closely even though the vendor played no part in the W3C process (which in a sense is actually good evidence that the spec is solid). Unless there's a mystery implementor shadowing every move we make, we're not going to be able to satisfy that requirement for XSLT 4.0: does this matter? My view is that a specification created using this community process is going to be of much higher quality than if a vendor like Saxonica had tried to go it alone, and that's what matters to the user community.

I'm confident that there will be other implementations in the future even it there aren't today: at this stage of the game, people want to see a standard set in stone before they start the expensive process of implementation[16]. There are even signs that AI technology is getting close to the point where given a good enough specification and test suite, the process of constructing an implementation can be largely automated. We're not there yet, but progress has been dramatic.

Meanwhile it remains very much the case that continuous implementation during development of the standard is necessary both to verify that features are practically implementable, and to provide early feedback on their usability.

And of course, the continuous and collaborative development of a comprehensive test suite is another key ingredient - but one which probably merits a separate paper.

References

[Kay 2020] Michael Kay. "A Proposal for XSLT 4.0". XML Prague, 2020. http://archive.xmlprague.cz/2020/files/xmlprague-2020-proceedings.pdf.

[Kay 2023] Michael Kay. "Schema-Aware Conversion of XML to JSON". Presented at Balisage: The Markup Conference 2022, Washington, DC, July 31 - August 4, 2023. In Proceedings of Balisage: The Markup Conference 2023. Balisage Series on Markup Technologies, vol. 28 (2023). doi:https://doi.org/10.4242/BalisageVol28.Kay01.

[Kay 2025] Michael Kay. Kay, Michael. "JNodes: a New Model for Navigating JSON Trees". Presented at Balisage: The Markup Conference 2025, Washington, DC, August 4 - 8, 2025. In Proceedings of Balisage: The Markup Conference 2025. Balisage Series on Markup Technologies, vol. 30 (2025). doi:https://doi.org/10.4242/BalisageVol30.Kay01.


[1] https://qt4cg.org/

[2] https://www.saxonica.com/products/latest.xml#saxon13

[3] https://github.com/qt4cg/qtspecs/pulls

[4] https://github.com/qt4cg/qt4tests

[5] https://github.com/qt4cg/xslt40-test

[6] https://blog.saxonica.com/mike/2026/05/101-things-in-QT4.html

[7] https://www.w3.org/groups/cg/

[8] https://qt4cg.org/specifications/xpath-functions-40/Overview.html

[9] https://qt4cg.org/specifications/xquery-40/xpath-40.html

[10] https://qt4cg.org/specifications/xslt-40/Overview.html

[11] https://qt4cg.org/specifications/xpath-datamodel-40/Overview.html

[12] https://github.com/qt4cg/qtspecs/issues

[13] https://github.com/qt4cg/qtspecs/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22Nice%20to%20have%22

[14] https://github.com/qt4cg/qtspecs/pull/2247

[15] "Citation needed", as they say on Wikipedia®. It is an unfortunate fact that we have no reliable data on how many people are using these technologies and what features they are using; we have to rely on anecdotal evidence.

[16] There are several projects currently striving to implement XSLT 3.0, although they are all still some way from that goal.

Michael Kay

Founder and Director of Innovation

Saxonica

Michael Kay is the founder and Director of Innovation at Saxonica. He was the original developer of the Saxon XSLT and XQuery processor, and was the editor of the XSLT 2.0 and 3.0 specifications. He was instrumental in establishing a W3C community group to create 4.0 versions of the XSLT, XQuery, and XPath languages. Since 2004 Saxonica has been a leading participant in both the development of the language specifications and in delivering high quality implementations of the standards. Michael is based in Reading, UK.