Balisage Paper: Graceful Tag Set Extension

Adoption

A user who has exactly the situation envisioned when a tag set was developed may well be able to simply use it. A user who wants to encode their system manuals in XML may find DocBook works well for them as published, and they will gain the added value of being able to use existing user interface layers on tools and formatting stylesheets.

Similarly, a user who want to send their journal articles to an archive or document repository may be required to use JATS (ANSI/NISO z39.96-2015), and may even be provided with guidelines that specify which of the JATS tag sets and how they should use optional features.

A user who wants to participate in an existing data interchange process may be required to use the tag set used by the existing participants regardless of comfort. For example, a user who wants to include their poster and pamphlet content in a publication locator service based on XML-tagged technical reports will have to find a way to tag those posters and pamphlets using the vocabulary used for the technical reports.

Adaptation

A community that wants to begin interchanging XML documents may find that there is no existing tag set and community of practice that exactly meets their needs. Some members of the community may be using XML, but if they have not worked together when developing their practices it is likely that they have different approaches. Even if the individual members have adopted public models, they may not have adopted the same public model.

The Standards Community is an example of a community that is currently working on developing a shared XML model for interchange of documents among the participants. For an excellent overview of this process, see NISO STS Project Overview and Update [Wheeler et al. 2016]. In this case, various members of the community already use DocBook-, DITA-, XHTML-, and JATS-based models, and at least one has done a TEI-based pilot project. None found any of the public models met their needs out-of-the-box; all adapted the models they had adopted. This community is now working to create an interchange tag set that will serve all of their needs. They are starting with a tag set created by one of the participants (ISO) that was developed by adopting and adapting JATS [ISO 2016]. This process is, we believe, typical of the way shared tag sets are being developed now.

Public models have been developed and documented with the assumption that they will be adapted. NIEM describes itself as a framework and provides tools for domains to use to develop information exchange packages [NIEM 2016]. DITA includes the Specialization feature, which enables users to extend the tag set and use DITA processors that are unaware of the extension [Eberlein et al. 2010]. The Text Encoding Initiative Guidelines describe clean and unclean modifications and provide a tool for creating extended TEI-based vocabularies [TEI 2016]. JATS documents how the tag sets can be modified [NCBI 2015] and provides terminology to identify and distinguish between JATS-Based and JATS-Conforming extensions [ANSI/NISO 2015].

As users adopt (by choice or fiat) a customizable model and begin to adapt that model to meet their needs, they are faced with decisions that may have far-reaching consequences. It is not uncommon for users to come to regret customization decisions made early in the adaption process. In some cases, there is considerable discussion of options, and a choice is made between what are known to be imperfect options. In other cases, however, the customizers do not even know that they are creating problems for themselves and their users down the line.

If your adapted tag set is for use in isolation, most of these guidelines are irrelevant to your project and usage. If you intend to craft or customize tools as needed and are unconcerned about how your adapted tag set will work with existing tools, others of these guidelines are irrelevant. If you are going to train all of the people who will create, manage, use, and archive your documents, others of these guidelines are irrelevant. If you and your documents are on a technologically isolated deserted island and expect to remain so, none of this matters to you; do what you want as you want.

Most tag set adapters want the documents that use their adopted/adapted tag set to play nicely with others. They want to able to store their documents in databases alongside documents tagged with the source tag set or other adaptations of it and to be able to search them all as one coherent collection. They want to be able to use tools such as editors with customized user interfaces by adding only those features needed for the new structures in their documents. They want to be able to use formatting and display tools for the existing documents by adding handling for any new structures, if that. (With a DITA specialization, even that should be unnecessary).

JATS Compatability Guidelines

We, the authors of this paper, have been inspired by the ways in which JATS is being extended, and we are occasionally surprised by problems people who have adapted JATS have reported. We have been drafting a set of Guidelines [Usdin et al. 2016] for people extending JATS to help them understand which adaptations will integrate gracefully into existing JATS environments and how to tell if an adaption might bite them later. To our surprise, this was not always obvious. Many types of adaptation that we initially assumed would be problematic seem to be fine, and a few types of changes that seem innocuous can create significant surprises at later stages of the document life cycle.

The principles articulated in this paper are based on the work done to develop the JATS Compatibility Guidelines [Usdin et al. 2016], and many of the examples are taken from JATS and the JATS Compatibility Guidelines. However, readers who intend to create a JATS-compatible tag set are referred to those Guidelines; this paper is not a substitute for those Guidelines. We also hope that the JATS work and the thought that went into creating those Guidelines is more widely applicable.

Respect the Semantics

Starting from first principles, when using or extending a tag set, respect the semantics of the starting structures. This should be obvious, but an amazing number of XML users think that they are doing no harm by repurposing an element or attribute they would not use for the original purpose.

They don’t call it tag abuse, but that is what it is. Sometimes blatant, sometimes with a story justifying bending the meaning of a structure for convenience, tag abuse is rarely a good short term strategy and virtually always a bad long term strategy.

Tag abuse is using an element or attribute for content for which it was not intended. Tags are abused when users are trying to control display. For example, it is common to use several empty <p> elements in HTML to produce some blank space on the screen. There are not several empty logical paragraphs in the document, this is tag abuse to achieve screen formatting. Similarly, using a block-quote element to emphasize instructions, making them stand out from the prose around them, may achieve an acceptable display at the cost of junking up searches for block-quotes and hiding the content from a search for instructions.

If you need to store the country in which some people live and you don’t use the phone number element for foreigners you could put their country names in the phone number element. We have seen this done. So, what happens when you start to validate phone numbers? Or when you decide that you can make phone calls across state lines and need a place to put the phone numbers for those people? Can your database list the countries for all authors? What about when a formatting engine inserts the usual punctuation for a phone number into those country names and displays them?

If your starting tag set has a tag called <state> for state or province do not create an attribute called @state with the possible values solid, liquid, gas, or plasma. Your state does not technically infringe on the original state, but it will confuse people. Call your attribute @state-of-matter or some such.

Sometimes tag abuse happens from a coincidence of names — when a new user does not check the semantics and is misled by a homophone. Oh, they think, I need an element for what the witness said at the trial and there is a <statement> element, not noticing that <statement> is defined as a logical proof or hypothesis.

Use the Same Style of Nesting/Recursion for Sections

There are, generally speaking, three styles of modeling nested sections in XML:

In the recursive model, sections contain sections, which can contain sections, which can contain sections. Display styling of the section headers is based on analysis of the location of the section in the section hierarchy.

In the nested-with-explicit-levels model, sections level 1 may contain sections level 2 which may contain sections level 3, etc.

In the non-nested with explicit levels model, sections level 1 may be followed by sections level 2 which may be followed by sections level 3, but these may come in any order and are not nested.

The section logic is fundamental to complex prose documents, and mixing section logic in the same environment creates the opportunity for significant confusion. People, and software, can get very confused if it is not clear, i.e., whether sections are nested or not; whether the level of nesting should be computed from the level of sections in which a section is contained or derived from the name of the section. Worst of all is a model in which sections that have explicitly named levels are sometimes nested at other levels. (Yes, this does occur in real documents.)

Maintain Distinction Between Elements and Attributes

In the XML world there are people who argue that the distinction between elements and attributes is arbitrary and that, since it is easy to transform one to the other using XSLT, vocabulary developers should feel free to use either at any time for any purpose. This may be so if the vocabulary is being developed in a vacuum, but if a new or modified vocabulary is intended to interoperate with another vocabulary, this is very much not so! While attributes are often used to control display, and their values may be used either to prompt selection of generated text or be displayed, their use in display is significantly different from element content. Similarly, while there are times when element content is not displayed, the default in most (text-based) applications is that element content is displayed to the reader. In most databases, attribute values are indexed, searched, and displayed differently from element content. Also, in most XML editing systems, attribute values are entered and displayed differently from element content.

If content in the source vocabulary is element content, keep it as element content. If it is attribute content, keep it as attribute content. If there is a need in a new vocabulary to change the form of content in a source vocabulary from element to attribute or vice versa, we recommend using a different name for the new structure and documenting its relationship to the content in the source vocabulary.

Whitespace Handling

In XML, some whitespace is significant and some is insignificant. How whitespace is handled has serious impact on the ability to re-use tools among documents in a heterogeneous collection. If elements in a tag set extension do not have the same whitespace handling properties as the display tools were developed to expect, there will be unfortunate (and in some cases surprising) effects on the display of the document content.

Three whitespace handling types are listed below. A compatible tag set extension must not change the whitespace handling type for any existing element.

ID, IDREF, and IDREFS

Rendering and behavior, especially the rending and behavior of links, is often dependent on the ID/IDREF relationship. If an attribute that has a type of ID in the source vocabulary is changed to any other type, rendering tools may not process the links appropriately.

Changing from IDREF to IDREFS or vice versa is not a concern. The number of pointers will not affect compatibility. Changing the direction of the pointer or obscuring the pointer is the concern here.

We have actually seen one instance in which a user reversed the uses of IDs and IDREFs, creating documents that looked similar to those in the source vocabulary. The result was chaotic; it turned out that the XSLT that created the HTML version of these documents relied on the ID/IDREF mechanism MOST of the time, but occasionally simply treated the attribute values as values. So, SOME of the links worked as expected and some did not. (On further thought, this is as much the fault of an inconsistent transformation as a surprising document; all of these links should probably have failed!)

Alternatives or Media-specific Content

In the world of prose documents, it is assumed that the reader should have access to all content. However, there are situations in which that is not the case. For example, it is common to provide several versions of the same graphical object: one for high resolution or full-screen display, one for display on small devices such as hand-helds, a thumbnail for navigation, and perhaps a very high resolution or black & white version for print. In counting the number of figures in a document, this figure should be counted once — not as many times as there are media- or use-specific versions — and only the most appropriate for the display media should be rendered. Similarly, it is becoming common for journals to publish author names both in the language and script of the journal and in the language and script of the author’s home environment. This person should be counted only once in specifying the number of authors of the paper and, more importantly, this paper should only count once when calculating the author’s influence.

Any structure in the original vocabulary that is provided to wrap two or more alternative structures, must be used in the same way in all compatible vocabularies.

EMPTY Elements versus Contenting-containing Ones

One obvious element differentiator was EMPTY elements versus those with #PCDATA, element, or mixed content. Element content is indeed unique, but data characters, mixed content, and EMPTY are all the same, since characters are, by definintion, optional in XML. An elmement with a #PCDATA model or mixed content may have nothing in it, and will look the same as an EMPTY element in the document. Thus, the following categories are uninteresting in this context:

Has Metadata

Some structures (whole documents, authors, boxed-text, appendices) may have metadata, and there are other structures that are unlikely to have metadata (italic, break, address-line). However, on analysis, we found that there are circumstances in which almost any structure could have metadata (at least an ID or IDREF that associates this structure with others), and that this does not affect interoperability as we were looking at it.

Is Metadata

In many tag sets, some elements are only used in the metadata of a document (journal in which published) while others are only used in the narrative text (figure). But in most tag sets there are many elements that can be used both to describe the document in which they occur and to describe other documents (copyright, digital identifiers, publication date), so this distinction is not just unimportant, it often changes over time.

Sections and Section-like Structures

It seemed intuitively obvious that an element that had the section structure in one vocabulary should have a section structure in a compatible vocabulary. That, for example, if a Boxed-text could contain not only paragraph-like structures but also nested headed sections in a source vocabulary, it should in any compatible vocabularies. But since those nested sections are, or could be, optional in the source vocabulary, documents without them can clearly be handled by the tools and formatter because we believe that a subset of an element model is always conforming. Thus, it is not necessary that compatible vocabularies allow nested sections in all of the places that the source vocabulary does.

Conversely, we considered that nested sections be allowed only in the places where they are allowed in the source vocabulary, and found that this, too, is not a requirement. If a tool or format is data driven (what in XSLT-speak is called push-processed), it should be able to accommodate sections that have the same style of sections as are already present in the vocabulary even in new locations.

Role in the Document

Structures can easily be grouped by their role in a document, and it is tempting to think that structures must play the same role in all document types in order to be compatible. We found that this is not so, and that while it might be interesting to group structures by their roles in documents, these roles do not seem to affect interoperability:

Attribute Value Types (other than ID and IDREF)

Even in a DTD, it is possible to type attribute values, and in XSD and RNG attribute value types can be quite strongly specified. We know (see above) that it is critical that attributes of type ID remain of type ID and that attributes of type IDREF or IDREFS remain of type IDREF or IDREFS in order for documents to be compatible. However, that leaves many other attribute types.

Some processing may be tied to specific values of attributes, and if none of the expected values are present the processing may fail. For example, if a formatter renders <styled-content view="GrIt"> as green italic, if that value is not present the formatter will not render the content in green and italic. However, we see no disruption from:

We came to the conclusion that most attribute typing is useful in the creation of correct documents as specified by the content creator, but is not essential to the storage, management, or rendering of the documents.