Balisage Paper: Can LLMs help with XML?

Anthropic and Claude

Claude.ai is the public interface to an LLM recently introduced by Anthropic. Its 3.0 release made an excellent first impression when I asked it to modernize the English of a Victorian-era translation of Philo of Alexandria. That task is a topic for another day, but while dumbing-down the XML documents to feed to the LLM I decided to try asking it not only to update the idiom, but to do some markup-specific tasks: First to preserve things like section numbers, MarkDown-style italics, etc; and later to add some markup — some from scratch as for sentences, some by mapping as for quotations.

On the whole, Claude did pretty well at such tasks, although it does make mistakes or lose track of one task while working on another. LLMs also produce different results each time even given the very same prompt.^[2]

Some training issues

LLMs scrape a lot of text for training. Training data selection and cleaning have strong (but hard to quantify) effects. Selection criteria are not always transparent, and even when they are, reviewing them or analyzing the effects is a gargantuan task. A number of confounding factors show up:

Claude's Constitution

Claude takes a novel approach to avoiding offensive or inappropriate responses via a published Constitution [Anthropic], a set of principles based on the UN Declaration of Human Rights and other inputs. The principles are formulated as maxims for the AI, and seem to be fed to it periodically behind the scenes. For example:

The principles factor into both training and response generation. It is not entirely clear to me why/how this works; one possibility is that certain turns of phrase or topics or styles are found in training to correlate with ethical criticisms (say, via reviews, retorts, community notes, etc.). Purely subjectively, I have found Claude produces fewer annoying, obsequious, or evasive results than some other LLMs, so I'm cautiously optimistic about this approach.

Information and structure loss in training data

As already mentioned, NLP systems often discard basic useful information. This leads to problems at higher levels:

Case distinguishes entirely different words more frequently than we tend to think (China, Mobile, Tangier, Chile, and countless others). Although case treatment has improved, many character set issues are still problematic, including servers asserting the wrong encoding for data and much software decoding (and corrupting) data without checking. LMM training data has the same problems. A reviewer noted that the Trésor de la langue française corpus discarded case on the ground that no lexicographer would cite a text without consulting the original; LLMs, alas, are not ideal lexicographers.

Lemmatization: Some NLP systems have aggressively stripped prefixes and suffixes. This can bring together closely-related tokens, but also brings together tokens that do not belong together (sing and singe).

Punctuation is much more complex than most NLP tokenizers can handle. I have seen systems tag and/or as three conjunctions rather than one; break URLs at all punctuation and then try to parse them as sentences (for which, calling / a conjunction ironically helps); take M*A*S*H as an equation; and be confused by many other simple phenomena.

Font changes such as bold, italics, monospace, and superscripts can matter a lot. Headings, lists, cross-references, and status as poetry or code correlate with drastic changes in grammar. LLMs sometimes learn other features that correlate with the lost ones, mitigating the damage; but it seems better not to suffer the loss in the first place.

Discarding such information poses problems both obvious and subtle. Case, punctuation, tokenization, and all these other phenomena inform downstream processing, from part of speech (PoS) to grammar to semantics.

Provenance and other metadata: Missing or incorrect metadata makes it harder to verify correctness. [Koplenig 2017] discusses similar problems in Google Books (hardly the only offender). [Nunberg 2009] discusses how difficult it is to pin down Google Books' actual date of composition, vs. dates of printing or scanning:

Non-content text: Google Ngrams used to report words such as digital and electronic in old works, because of boilerplate notices about the digitized edition's status — even Google's own code didn't account for this. A subtler but similar problem occurs in editions with added headings, notes, etc., which may easily be misconstrued as part of an author's work. Page headers and footers in scanned texts are another infamous case.

Selection issues: Attempts to mitigate bias or inappropriate text may use very blunt instruments, such as AOL's list of censored words including breast and many other terms [Chicago Tribune 1995]. Much more recent and commonly-used lists such as badwords (https://www.cs.cmu.edu/~biglou/resources/bad-words.txt) and Shutterstock's LDNOOBW (aka List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words) [Simonite 2021] remain astonishingly uneven, with no discernible rationale for what is or isn't included.^[3]

More aggressive censoring may reject on parts of words or obvious spelling deformations. Of course this massively increases false positives. This is known as the Scunthorpe problem [Wikipedia], since the eponymous town was censored (by AOL in 1996 and Google in the early 2000s) due to a substring of its name. If asked, Claude can describe the Scunthorpe problem, so evidently was not so naïve as to unconditionally censor the string during training. ChatGPT describes the town but says nothing of the censorship problem unless pressed more specifically.

Less commonly noted is that naive censoring is as easily applied in training as later. That may seem tempting in order to keep bad words entirely out of language models (see [Bender 2021]), but has obvious unintended consequences.

Challenges specific to long texts

Like other LLMs, Claude may not work as well or consistently across a very long text. LLMs may wander off task or occasionally crash long into a paid task. So it is wise to break documents into smaller parts. On the other hand, that means that for each given part there is less to get the model into the right context, and the chunks may be treated differently. For example, inserted markup might show shifting conventions, and pronouns whose antecedents are in a different chunk may be misconstrued. For example, one chunk of Philo I had Claude modernize came back with they at the beginning replaced by Egypt (which was not relevant at all).

This problem recalls extensive discussions of chunking in the hypertext world. [Bernstein 2010] and [Marshall 1989] detail the debates over breaking information into small pieces (advocated by card sharks) vs. supporting large units and precise linking (for holy scrollers). This conflict can still be seen all over the Web despite size and bandwidth issues having almost vanished. Indeed, as of 2022 median page size (including figures, CSS, etc.) had risen to 2.2MB [HTTP 2022, Chapter IV.21], making the text a tiny fraction. Breaking a book into tiny chunks may seem quaint when the entire book is shorter than the CSS and Javascript.

If one must break a long text into parts, breaking by logical parts (say, sections of one level or another) may be much better than by size even though logical units may have widely varying sizes. Breaking by size also tends to group headings with what came before, not what comes after (they are shorter, so less likely to trigger a size limit).

One compromise for LLMs is to pass some of the already-handled text to the LLM again, as preamble to the next part. Another is to reiterate the prompt each time, and make it very explicit about what markup to insert. However, LLMs are probabilistic so there will be unevenness. Claude has its own thoughts on this process, for which see below.

What LLMs can and cannot do

Despite huge differences, issues seen with HMMs, word2vec, and prior statistical models presage the strengths and weaknesses of current LLMs. Something fundamental about language is captured — the randomly-generated texts are in the right language in some sense; word2vec's semantic arithmetic is capturing something real. Yet the output need not make sense — nothing grounds it in specific facts in the world. None of these models has understanding; they merely produce probable results. Therefore, with training on data that is mostly accurate LLMs should generate data that is also mostly accurate — probable token sequences mimic the training data. But this is a model of human behavior, not the real thing. As [Searle 2002, p. 16] points out:

Searle's Chinese Room thought experiment has become classic in this regard (see [Cole 2020]): Imagine an AI proficient at answering questions in Chinese. It receives written questions, and sends back convincing responses. Searle posed the question whether such a device can be said to understand Chinese. For those tempted to say yes, Searle points out that if such a device could exist Searle himself could just execute the same algorithms using lower tech such as file cabinets, dictionaries, etc.; and that we would not say he therefore understands Chinese. Leibnitz, Descartes, and many others have made similar points. For example, Leibnitz wrote of a Mill (Leibnitz 1714, section 17; cf Cole 2022):

The state of the mill is precisely the state of LLMs. LLMs are (at most) Turing Machines: Their operation is governed by a deterministic automaton (the CPU). Even with infinite memory this would remain true: CPUs are finite-state machines with finite precision math; their tables and code are the tape.^[5] Claude also says There's no scientific basis for claims that large language models (LLMs) like myself are 'beyond' Turing machines. – but then, a hypercomputing AI might say that anyway.

LLMs incrementally adjust their parameters (their DNA, if you will) to evolve them to maximize one goal, known in AI as a reward function. The goal is generating token sequences that seem probable to humans. Success leads to survival of the best LLMs. As noted earlier, this reward function is not the same as that of making true statements. If a sufficiently overwhelming fraction of the training data is true, there will be some correlation with truth (and indeed, we perceive LLMs as having such a correlation). But that correlation is indirect and, crucially, not reliable. Aiming at a different goal does not preclude truth -- but it does preclude reliable truth; LLMs are not a truth-preserving mechanism.

Thinkers as diverse as Charles Darwin, Patricia Churchland, and Alvin Plantinga (among others) have pointed out a parallel problem: Evolution is commonly (though not necessaily) associated with naturalism: the claim that there is no causation beyond material physics (in effect, nothing resembling God — advanced aliens don't help, because they presumably evolved similarly). Like LLMs, evolution has a reward function: survival. And like LLMs, the reward function is not the same as accuracy. Darwin noted this briefly in a letter (some suggest he was speaking only of non-scientific matters, though I see no suggestion of that in the letter) [Darwin 1881]:

Churchland, an eminent neurophilosopher, put it thus in Epistemology in the Age of Neuroscience [Churchland 1987]:

The crucial step is that our reasoning mechanisms also evolved, by hypothesis naturalistically. And thus our reasoning should produce results in keeping with the reward function: conclusions that tend to keep us alive, not to reliably discover truth. Our reasoning and our most plausible arguments about reasoning are subject to the same flaws. Plantinga's version of this argument [Plantinga 1993] is known as the Evolutionary Argument Against Naturalism (EAAN). The point is not that all our beliefs must be false, or even any particular share of them. Rather, the point is that reasoning capabilities arising from naturalistic evolution cannot justify confidence that they, themselves, are reliable. This has of course led to heated argument with regard to humans; but it is certainly the exact state of affairs we know to be true of LLMs. The primary reason for AI hallucination is the same: a reward function quite distinct from truth or accuracy.

Consider for a moment a future where LLMs continue to grow, improve, and become more and more convincing. They will likely begin to seem self-aware; seem to have beliefs about the world; seem to reason. Whether or not strong AI is true (the belief that AI will eventually build complete persons; that there is nothing fundamentally different about persons), they will arrive at the condition that these thinkers have described for us — yet remain unable to know whether or when they are producing truth. We can (and do) feed them nontruths, and when we do they cannot know the difference. If they find apparent contradictions, they cannot know which side to take or even whether the contradiction is real or merely apparent (whether due to equivocation or other missteps). Conceivably, such simulations have already reached that point. Conceivably, they are us.

Part of speech tagging

I asked Claude to assign part of speech tags to the first sentence of the Brown Corpus [Francis 1979]:

The result differs from the curated Brown Corpus tagging, but is quite good, as is trying for the same result in XML:

Sentence and quotation

In e-literature it is common to mark up sentences and quotations (and sometimes smaller units such as clauses). An LLM has to have implicit knowledge of these to do its other work, but can it identify and even mark up those units reliably? Let's see:

Named entities

Continuing the conversation just described, I tried a slightly harder case, namely tagging the semantic category of demonyms (names of demographic groups). Claude did fine, as it has on several other NER tasks I've tried:

More on quotations

In the wild sometimes quotations aren't quoted. Claude made a reasonable attempt at suggesting those:

Quotes are often discontiguous. For example, they are often broken up by speaker tags such as she said. I spent quite some time trying to get Claude to co-index the parts of individual speeches. At first it had trouble even generating well-formed quote markup, omitting </q> in multi-paragraph speech. This was probably influenced by the conventions for close quotation marks in fiction. Oddly, Claude explained that it had done that; so it knew, but didn't think it worth attending to:

When pressed, Claude fixed the end-tags. I then asked it to co-index the parts of quotes. Claude first suggested this odd solution:

After a few more similar fails, I asked Can you repeat that, but add a co-indexing attribute …? At first it serial numbered the quotes (and tried to explain why that's useful). With a few more tries it got to:

This is at least syntactically the right idea, but it only used one index value. For now co-indexing seems hard to achieve. At this point I started a new conversation, and gave Claude the final version as just shown. I then asked questions about it. It counted 4 physical/syntactic quotes, 1 logical quote, and that the speaker is only identified as she.

This last response tells us a lot. It's nice that Claude's exploration of the world taught it that TEI is the right choice here, and that it can apply it. It's also nice that it took the initiative and didn't just reply yes to my question, but named the spec and applied it.

Quote identification

Identifying the source for quotations would be very handy. This is a fraught problem because of translation, paraphrase, and spurious attribution, which is rampant and often repeated in places LLMs may scrape for training (making it a particularly telling example of the grounding problem). The quotation

is commonly attributed to Dante. A Google search found the phrase so attributed at brainyquote.com, goodreads.com, elevatesociety.com, quotefancy.com, sacredtouches.com, wordsandquotes.com, and many more. Claude cannot swim against the tide of frequency it takes as proxy for truth:

This is pure fabrication. In a fresh conversation, Claude translates the Italian lines to There you will see the soldiery of paradise / In two divisions, one of them in those aspects / That you will see at the final judgment, and without being asked, correctly identifies its source as Canto XXX (it is lines 28-45) — not Canto XXI as it suggested earlier. I find nothing like the queried quotation in either Canto, nor in Canto XXVII which Claude suggested next.^[7] I later tried the same prompt in a fresh conversation, but suggested it was commonly attributed to Mark Twain. Claude said the following (which also does not check out):

ChatGPT instead suggested Gaston Bachelard's book The Psychoanalysis of Fire. As expected, frequency trumps truth.

Bibliography entries

Converting bibliographies can be tedious, from managing tag sets with wildly varying names, to granularity and category differences, to personal name formats. I copied the bibliography from an Oxygen formatted view of this article, and pasted it to a plain-text editor (no tags at all; not even blank lines between entries, though each started a new line). Claude produced thoroughly tagged entries. Name parts, publisher locations, volume vs. issue vs. page, etc.

Next, I asked it to take the entries whose first authors' surname begin with letter from A to C, and create me a new version in BiBTEX. It did a reasonable job:

Claude also did well on questions about the marked-up results, getting all the following correct:

The operations on bibliography entries seem to me the most practically useful of all the tasks. Claude can detect and generate good bibliography markup from plain text, translate between formats, handle the vagaries of names, and answer fairly complex structure-dependent questions about the results.

MarkDown to HTML

A sample Markdown file with a wide range of constructs (nested lists, tables, deflists, pre, and various inlines and headings), translated just fine to HTML. Interestingly, Claude chose <strong> and <em> rather than <i> and <b>, and wrote a reasonable stylesheet in the header (not inline @style). It was also able to operate on the results, such as to grab me all the content from 2nd-level lists.

When asked how many characters of content occur within inline elements? it produced an overall total and counts by element type. Unfortunately the counts were wrong (it is commonly reported that LLMs fail badly at arithmetic). When asked to also include the markup of those elements, it gave larger though still incorrect counts. However, asked to count element instances, it seemed reliable. Without asking it also put up Python code to do character counting by element type.

Transliteration

Claude did fine at translating Greek (which included diacritics) to Latin characters, When asked about the scheme it chose, it gave a plausible but incomplete discussion. When pressed about conventions it mentioned the ISO 843 and ALA-LC standards. It didn't mention Betacode at all, but generated it when specifically asked.

Dictionary entry markup

Digitization of the Oxford English Dictionary showed how complex dictionary entries (and their markup) can be [Berg 1988]. I found a png image of an OED sample entry at https://guides.library.uab.edu/Milton/entries, and OCRed it; the result had white space and punctuation intact, but no other markup:

literate, adj. and n.
...
Pronunciation: Brit OP lit(a)rat/, U.S. OP licla rat/
Forms: late Middle English-160os litterate,
...

My initial prompt, Here's a dictionary entry, in 'plain text'. Can you mark it up for me? produced HTML that looked nice but was otherwise very poor. Prompting with This time, try for semantic/descriptive markup. You can just pick appropriate mnemonic names, or use TEI. worked pretty well (the sample below is edited to save space). When I asked it to move the sense_number content up to an n attribute= and tag all the dates, a new version was added to the artifact pane.

<entry>
  <form> <orth>literate</orth> <pos>adj. and n.</pos> </form>
  <pronunciations>
    <pron type="BrE">ˈlɪt(ə)rət</pron><pron type="AmE">ˈlɪdərət</pron>
  </pronunciations>
  <etymology>
    <etym_lang>A borrowing from Latin.</etym_lang>
    <etym_source>Latin litteratus</etym_source>
    <etym_detail>
      &lt; classical Latin litteratus marked or inscribed with letters, well versed in
      literature, cultured, erudite,...</etym_detail>
    <usage_note>...</usage_note></etymology>
  <form_history>
    <form_variant>late Middle English-1600s litterate</form_variant>
    <form_variant>1500s lytterate</form_variant>...</form_history>
  <sense type="adjective"><sense_number>A. 1.</sense_number>
    <definition><def_text>Of a person, society, etc.: acquainted with letters
      or literature; erudite, learned, lettered. Also with in (a subject). Now rare.</def_text>
      <usage_note>In later use coloured by sense A. 1b.</usage_note>
    </definition>...

Spelling correction

This should be easy for an LLM. Given a sample from the Holbrook spelling error database at https://www.dcs.bbk.ac.uk/~ROGER/holbrook-tagged.dat, Claude corrected the misspelled words and several other errors. Asking it only to do spelling errors per se avoided most but not all of the other cases. It is also able to do TEI-style sic/corr markup, and add a type attribute to categorize each error as 'spelling,' 'word-choice,' 'form' (for wrong endings), 'case,' or 'punc' (for punctuation).:

The ability to mark up its changes seems to me quite valuable. Adding corrections to an established text archive or corpus can be fraught, and marking them up makes the history explicit and easy to review and improve.

Understanding ThML/DC header data

For several of these tests I used text from Philo of Alexandria [Yonge 1854-1855], downloaded from ccel.org in ThML, an extension of HTML with support for Dublin Core (DC) header metadata, inline Biblical references, etc. First I gave it the header (as raw ThML). Without further prompting, it said:

I asked how it knew it was ThML, and it cited several specific element type names. Pushing further on the DC tags, it recognized the relationship of the sub attribute on the DC tags to USMARC Relator Codes, identified the ones in use, and updated the values above to the standard 3-letter codes. I did not dig into analysis of the main text, since that would have to focus on the content rather than the markup.