Balisage Paper: Ensuring XML Quality and Compatibility in Large Collections That Span Decades of Content

Discoverability

Structured XML allows readers to find content more precisely by allowing for advanced searches beyond simple text queries. The structural cues in the content and the metadata enable precise searching. Well-formatted XML documents allow indexing tools and applications to effectively parse and better identify relevant data faster, which is particularly valuable in large datasets and digital libraries.

Content Interchange

Today, the same information is reused in many forms and is expected to transfer seamlessly between systems and platforms. Consistent and accurate XML structures are crucial for this content interchange. XML’s standardization allows disparate systems to communicate easily, ensuring that data retains its meaning and structure when transferred. Interoperability is fundamental for data sharing across organizational boundaries.

Accessibility

Accessibility to information for people with disabilities is important, and it is also often legally required. Accurate structuring and tagging of XML does much of the work to make content accessible and allows much more to be done. For example, images of tables and math that are not encoded are inaccessible to visually disabled users; XML provides the mechanisms needed to encode and structure tables and math so that they are readable by software readers to ensure they are accessible to those with visual and other disabilities.

Emerging and Future Applications

Content discoverability, interchange, and accessibility are well-established applications in the scholarly publishing arena; however, many new XML-dependent applications are currently in development, and we don’t know all the ones that are coming along. Applications in the wings include various AI applications as well, to meet the challenges of research integrity.

Changing Standards

Standards evolve. XML has been an industry standard in scholarly publishing since at least 2003, when the National Library of Medicine (NLM) introduced the NLM DTD Suite. [Rosenblum 2010] Focusing on the JATS standard alone, there have been seven core iterations since 2003 [Wikipedia], and several intermediate versions (Table I). That alone accounts for significant variation over time. Some of the earlier versions are not totally compatible with the current versions and are also missing desirable features that were implemented later.

Interpretation of Standards and Errors in Execution

The various standard versions are complex, and not everyone is fully conversant with all the elements and attributes or even has the same understanding of how to interpret them. Because publishers often work with multiple parties and vendors to produce XML based on the standard at the time, variances in tagging interpretation and application, differences in the specifications, and human error all contribute to inconsistencies. Production errors, interpretations, and variations in production support can all lead to discrepancies in the XML files over decades of publishing.

Archival Content is Often Not Consistent with Current Content

Publishers with significant history have likely published content in various NLM/JATS specification iterations. Yet few publishers do a systematic review of their content to analyze and identify key metrics about their source files and corresponding digital assets (e.g., .jpg, .gif, .XLSX, .zip, .tif, .mov, etc.) to bring them up to a common standard; a variety of standards continue to proliferate.

Some of these earlier materials may be following obsolete standards. Publishers may be missing out on important functionality by not updating to the current standard. Understanding what is in a publisher’s legacy archives is a critical first step.

A major factor for many publishers is that not all their content is in XML. Depending on the investment in publishing technology, legacy content could comprise PDFs, PDFs with XML header information, or XML files in an outdated DTD, along with dozens of other possibilities.

Missing DOIs

Web hosting platforms such as Silverchair, HighWire, and Atypon use the Digital Object Identifiers (DOI) as a unique identifier for the article or paper. If the DOI is missing or not valid, the content may not load properly, or may not be loaded to the platform at all. If the DOI is not unique, subsequent loads might overwrite earlier content or cause other problems. Missing DOIs found at load time can often delay the whole process.

Identifying these problems in advance provides time to fix the problem before the crunch. Missing identifiers can be found with lookups against publishers’ records or standard databases like Crossref. Or if the DOIs do not exist, they can be created. But all this is best done earlier rather than at platform load time.

Cross-Reference Errors

Missing and incorrect cross-references (<xref>s) are common. Examples are image files that are not referred to in the text, or references in the text to images that are not found. Incorrect and empty <xref> tags must be resolved, or they will not properly render on hosting platforms and will not find the items referred to.

Following are some examples in a recent analysis:

Invalid Assets

Aside from the text, there are other objects (assets) like images, math, video, etc., which typically need to be incorporated into the displayed document. Requirements for assets vary across hosting platforms, which may have different file formats requirements, naming conventions, and other specifics.

Following are common examples of issues with assets. While these problems do not appear as XML errors, they demonstrate the necessity of analyzing archival and current content and the supporting files that make a journal article or book chapter complete:

Other Structural Problems

XML files can be valid and parse cleanly, and still contain structural issues that render the content not fully usable and findable.

Unusual but True Findings in JATS

Until a collection is deeply mined and analyzed, it’s hard to imagine the issues that might be found in a large collection spanning many decades. Just when we think we’ve seen it all, we find something new:

Identifying What You’ve Got

The first step sounds simple, but sometimes isn’t – identify all your content and make sure it’s available to analyze. The publisher’s collection, organized in hierarchical folders and files, undergoes an initial process phase to organize the content into units, extract relevant metadata from the files, and verify completeness. A unit might be an article, a journal issue, a manuscript, conference proceedings, or anything else. For books, either each chapter or an entire book might be considered a unit. This processing phase also inventories digital assets such as .jpg, .gif, .XLSX, .r, .JSON, and other file types. The technologies used include DOM, XPath, regular expressions, custom algorithms, Exif tools, PDF tools, and Ghostscript.

Architecting the Process

Because of the sheer amount of content that might need to be audited, often a million pages or more, it’s important that software is architected to be multithreaded, restartable, and incremental.

XML Parsing and Validation

The next phase involves health checks on digital assets including validating XML files. Software performs semantic checks on XML files at the unit level, journal title level, and across the collection. Additionally, the relationships between the XML files and digital assets are analyzed and verified.

Meeting Business Rules: DCL’s Global QC Approach

Aside from the technical details defined by the DTD or Schema, there are other publisher- or journal-specific rules on content that organizations should check to ensure the content base is consistent across the collection. We call those business rules; these are rules that apply to a particular business or situation.

Following are just a few examples from DCL clients to illustrate the kinds of rules that might be applied:

Business rules can vary from client to client and even between one project and another, thus architecting a configurable system that defines and analyzes these rules across large content sets is essential.

DCL’s Global QC is a platform to facilitate defining business rules without programming and applying them across a large corpus (Figure 1). Quality checks can be performed on individual XML files or throughout the corpus. The checks can be restricted to DTDs or XML schemas, configured to exclude/include specific nodes within the content, be conditional based on business scenarios, and easily be grouped together into sets for consistent handling. We’ve also built an intuitive user interface to create, modify, and delete verification checks and apply them to the content to address common scenarios and data anomalies. Each individual or related set of checks may be assigned severity levels and customized error messaging to enable efficient downstream error handling.

Figure 1: Global QC by DCL is a configurable tool to validate large XML collections.

Seamless Interchange

Quality XML is crucial for content access, interpretation, and exchange. If publishers routinely analyzed their collections, the anomalies, errors, and issues that have been lurking might have been revealed earlier, which would have already improved interchange and discoverability. This part is not new; what is new is that we don’t know all the places we want content to go or all the new ways we will want to interchange. Having content rigorously standardized ensures that you will be ready for what comes next.

Research Integrity

Scholarly publishing is struggling with problematic papers getting into publishing workflows via papermills and citation cartels. XML is increasingly used to effectively identify potential issues related to research integrity by making it easier to verify the accuracy of references and author names and affiliations with third-party databases (e.g., Crossref, Scopus, Web of Science, CCC, ORCID).

XML tagging and metadata provide a structured framework for analyzing journal articles at scale, enabling publishers to effectively identify potential issues related to research integrity. DCL is building a library of automated research integrity checks to flag and report suspected issues at scale. Some integrity checks involve traditional XML analysis and validation, but other tools are also used.

Adjacent technology, tools, and processes invoked for XML analysis, such as entity extraction, machine learning, and natural language processing, are employed for automated analysis, data mining, and reporting across an entire collection. Some integrity checks we’ve been applying or studying include

AI Initiatives

The rapid growth of AI in the research community has given new additional importance to structured content. While AI development is still in its infancy, it’s becoming clear that AI systems will produce better answers when it gets more clues that well-structured data and metadata provide. Adopting a consistent and accurate XML structure ensures that organizations will have content that will be AI-ready and helps preserve the value of their content assets. This will become additionally important when combining RAG (Retrieval-Augmented Generation) with structured content in an LLM (large language model) to improve the accuracy, timeliness, and factual grounding of LLM output.

Structured content provides potential advantages for finetuning an LLM such as OpenAI’s GPT model to refine its knowledge based on a specific domain. [Abbey 2024] By training on verified and structured data, the AI model can better parse and process information and ultimately minimize hallucinations.

Structured content has many potential advantages over unstructured content when it comes to training and finetuning language models: