Discovering a Long Dormant Bug in XML 1.0 Balisage: The Markup Conference 2026 August 3-7, 2026 While investigating a two decade old TODO comment, an even older problem in the XML 1.0 specification was uncovered. The specification is contradictory about character escaping inside system literals. This contradiction has remained unchanged from the first edition in 1998 until now. Fortunately, parsers seem to have settled on a single consistent interpretation. Unfortunately, the XML Core Working Group has disbanded so it's unclear if an erratum can be issued. Elliotte Harold When not laboring in his secret identity of a mild-mannered software developer, Elliotte Rusty Harold lives in a secret mountaintop laboratory on a large island off the East Coast of the United States with his wife Beth and dog Cream and two cats, Sheldon and Neutrino. He's an avid birder and entomology enthusiast. His most recent books are Java Network Programming, 4th edition, and the JavaMail API, both from O'Reilly. He's the creator of the XOM library for processing XML with Java, the current maintainer of the Jaxen XPath engine, an Apache Maven committer, and a member of the Xerces PMC. elharo@ibiblio.org https://www.elharo.com/ https://orcid.org/0009-0001-3159-0192 Copyright 2026 Elliotte Rusty Harold
Introduction Bald-faced Hornet Queens hibernate through the winter to found a new colony in the spring.
Hibernating Queen A hibernating Bald-faced Hornet Queen sleeping through the winter in a cavity she has chewed into a log
Cicadas live underground for up to 17 years before they emerge by the billions in a thunderous summer night symphony.
17-year Cicada Pharaoh Cicada Magicicada septendecim, Brood II, from the 2013 emergence
Is it possible for a bug to lie dormant for almost 30? Yes, yes it is. Apparently, I first noticed the problem in 2005 since that's when I added a to-do comment into the XOMHandler class indicating that the lines of code for managing system identifiers in DOCTYPE declarations needed a little bit more thought: [] if (publicID != null) { internalDTDSubset.append(" PUBLIC \""); internalDTDSubset.append(publicID); internalDTDSubset.append("\" \""); internalDTDSubset.append(systemID); } else { // need to escape system ID???? internalDTDSubset.append(" SYSTEM \""); internalDTDSubset.append(systemID); } Then I promptly forgot about it. More recently, I ran GitHub Copilot across the entire XOM codebase, specifically instructing it to find all the assorted to-do comments and file issues for each one. [] Many of the comments it found were moot, already fixed, or just not all that important, but a couple were actually significant, and this was one of them. []
The Problem Looking at this code two decades later, I didn't recall exactly how system identifiers were supposed to be escaped so I dug into the XML specification. This is a little harder than it used to be because the W3C really tries to hide the actual original XML 1.0 specification from 1998. But I did finally find it. And when I did, here's what I saw. According to section 2.4:
The ampersand character (&) and the left angle bracket (<) MUST NOT appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, or a CDATA section. If they are needed elsewhere, they MUST be escaped using either numeric character references or the strings "&amp;" and "&lt;" respectively.
The same text appears unchanged in the Fifth Edition [] and in the XML 1.1 Second edition. [] A system literal is clearly not a comment, not a processing instruction, not a CDATA section, and not a markup delimiter. Therefore, an ampersand or less than sign that appears inside one should be escaped. Crystal clear, right? Unfortunately, if you happen to also read section 2.3, you'll see this:
Literal data is any quoted string not containing the quotation mark used as a delimiter for that string. Literals are used for specifying the content of internal entities (EntityValue), the values of attributes (AttValue), and external identifiers (SystemLiteral). Note that a SystemLiteral can be parsed without scanning for markup.
Emphasis added. This isn't an academic point. Less than signs don't appear in all that many URLs, but ampersands appear in URLs routinely, especially ones that have query string components. Imagine a parser reading a DTD that encounters the URL http://www.example.com/foo?bar=baz&wik=wek How should it decode that system identifier? As http://www.example.com/foo?bar=baz&wik=wek or as http://www.example.com/foo?bar=baz&amp;wik=wek These aren't the same. And if you think that's obvious, what about: http://www.example.com/foo?bar=baz&amp;wik=wek Is there one algorithm that covers both cases? Do we just have to special case URL decoding based on what comes after the ampersand in a system literal? Fortunately, we can fall back on the BNF. According to production 11: [] [11] SystemLiteral ::= ('"' [^"]* '"') | ("'" [^']* "'") This does allow & and < to appear unescaped in system identifiers. If you're uncertain about that, compare this production with the similar and immediately preceding production for attribute values which does explicitly rule out ampersands and less than signs: [] [10] AttValue ::= '"' ([^<&"] | Reference)* '"' | "'" ([^<&'] | Reference)* "'" It seems the preponderance of the evidence does point toward not escaping ampersands and less than signs in URLs. The text that suggests otherwise is simply and error, and I'll come back to that. But first…
How do different parsers behave? As we know all too well, parsers don't always adhere to the strict letter of the XML specification, even when it's much less ambiguous than this. [] So I typed up a couple of very simple documents, one that escaped system literals in the internal DTD subset and one that didn't. First, we need the simple DTD shown in this listing: <!ELEMENT root EMPTY> What matters is not so much what's in this DTD, but what its name is. This is saved in a file named root&nothing.dtd. The ampersand is the critical bit. Do we reference that as a raw & as in the first listing below or as an escaped &amp; as in the second listing below? <!DOCTYPE root SYSTEM "root&nothing.dtd"> <root /> <!DOCTYPE root SYSTEM "root&amp;nothing.dtd"> <root /> A parser should find the DTD with one of these documents and fail to find it with the other. To find out which, I validated each with some common parsers to see which they preferred. The results are summarized in . For non-validating parsers like expat, we expect a parser that does not unescape system literals to report both examples as well-formed and one that does unescape to report the first as malformed.
Parser system literal handling
Parser System Literal Treatment
JDK 8+ com.sun.xml.internal.stream.XMLInputFactoryImpl Tested in OpenJDK 1.8.0_372, Eclipse Adoptium 11.0.16.1, OpenJDK 17.0.18, OpenJDK 25.0.2 Read literally
JDK 8+ com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl Tested in OpenJDK 1.8.0_372, Eclipse Adoptium 11.0.16.1, OpenJDK 17.0.18, OpenJDK 25.0.2 Read literally
libxml2 version 2.10.4 Read literally
Expat 2.7.3 (Python 3.14) Read literally
The XML Conformance Test Suite The W3C XML Conformance Test Suite is a bit of an afterthought. [] It post-dates the original spec by some years. Nonetheless, it has some claim to normativity. Does it address this case? Unfortunately, no. There are no test cases in the suite (so far as I can find) that contain ampersands or less than signs in system literals. This is neither a negative nor a positive case. It is perhaps worth noting that the test suite does contains explicit negative ("not well-formed") tests that verify parsers reject unescaped < and & characters within attribute values as required by production 10. (ibm-not-wf-P10-ibm10n01.xml and ibm-not-wf-P10-ibm10n02.xml).
A Proposed Erratum The text of the spec "The ampersand character (&) and the left angle bracket (<) MUST NOT appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, or a CDATA section." is not a well-formedness constraint and therefore not a requirement. The BNF takes precedence and fortunately agrees with parser behavior. Thus, we can classify this as a bug in the spec. We can add the missing case and issue an erratum:
"The ampersand character (&) and the left angle bracket (<) MUST NOT appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, system literal, or a CDATA section."
Or can we? Errata for the XML 1.0 and 1.1 specifications are approved and issued by the W3C XML Core Working Group. Unfortunately, this working group dissolved 10 years ago. [] I'm not sure it's worth reconstituting it just to do this, but it does feel like this is a hole in the W3C process. XML 1.0 is still very much alive and still a very successful technology. Developers, both human and machine, consume the specification and rely on it to answer questions like this. And this is clearly a less than ideal situation. Just maybe a test case can be added to the XML Conformance Test Suite that covers this. It's a little tricky due to different rules about characters allowed in file names on multiple platforms, and the desire not to load the test case DTDs from live web servers. Nonetheless, this is only a minor hurdle and should be manageable. The public-xml-testsuite mailing list doesn't seem to have had any activity since 2014, and my attempt to subscribe bounced with "Address not found", but perhaps it can be resurrected. [] Specifications in active use in the world today need some form of support for when new problems are uncovered or for when the external situation changes in a way that somehow renders the specification now inaccurate. Consider, for example, time zone shifts, new currencies, or even new laws that somehow cause problems for existing specs. I'm not aware of any cases where these have happened to XML, but I know of many cases where these have happened in other arenas of technology. While the stability of the XML specification is of critical importance, we should be prepared for the possibility that the specification is not just less than perfect, but contradictory and unimplementable. This is highly unlikely to happen in the core parts of the specification that have been battle tested for nearly three decades now. However, as this example demonstrates, there are still pieces no one's looked at closely enough. We should have a way to fix these.
Bibliography Bray, Tim, Jean Paoli, C. M. Sperberg-McQueen, Eve Maler, François Yergeau. eds. Extensible Markup Language (XML) 1.0 (Fifth Edition). W3C Recommendation, 26 November 2008. https://www.w3.org/TR/1998/REC-xml-19980210. Bray, Tim, Jean Paoli, C. M. Sperberg-McQueen, Eve Maler, François Yergeau. eds. Extensible Markup Language (XML) 1.0 (Fifth Edition). W3C Recommendation, 26 November 2008. https://www.w3.org/TR/2008/REC-xml-20081126/. Bray, Tim, Jean Paoli, C. M. Sperberg-McQueen, Eve Maler, François Yergeau, John Cowan. eds. Extensible Markup Language (XML) 1.1 (Second Edition). W3C Recommendation, 16 August 2006. http://www.w3.org/TR/2006/REC-xml11-20060816. Harold, Elliotte Rusty. Decided to retain all entity declarations in internal DTD subset . 16 January 2005. https://github.com/elharo/xom/commit/89f64a95002d918a48350e8cf1f16911b2b523e5. Harold, Elliotte Rusty. Convert TODO and ???? into issues #340. 31 March 2026. https://github.com/elharo/xom/issues/340. GitHub Copilot. XOMHandler: should system IDs in DTD declarations be escaped? #358. 31 March 2026. https://github.com/elharo/xom/issues/358. Quin, Liam. XML Core Working Group Public Page. 10 August 2017. https://www.w3.org/XML/Core/. World Wide Web Consortium. XML Conformance Test Suite. 23 September 2013. https://www.w3.org/XML/Test/. World Wide Web Consortium. public-xml-testsuite mailing list. 13 July 2023. https://lists.w3.org/Archives/Public/public-xml-testsuite/.