Introduction

The dita-std (DITA) defines a unique approach to the definition and management of XML document vocabularies. Rather than defining a monolithic set of element types and attributes that can then be modified or added to without constraint, DITA defines a formal architecture consisting of a set of invariant base vocabulary "modules" comprising the base standard DITA vocabulary. These modules are invariant in the sense that they are not intended to be modified directly but only through specific configuration and extension facilities as defined by the DITA standard. In particular, modules may not be modified in a way that makes them less constrained than the base definition. All copies of a given module version in time should be identical.

A given set of modules represents a unique "DITA document type". Two DITA documents that use the same modules have the same DITA document type. Note that this notion of "DITA document type" is independent of the use of any particular document grammar associated with the document: it is only necessary for a document to declare the set of modules it uses, which is done with the DITA-defined @domains attribute on the document root element. The value of the @domains attribute is simply a list of modules: <topic domains="(topic topic) (topic hi-d) (topic pr-d)">.

Likewise, each DITA element declares it's relationship to the base DITA vocabulary through the the DITA-defined @class attribute, which enumerates the element's ancestry as a sequence of module/tagname pairs: <steps class="- topic/ol task/steps ">. For the <steps> element, the @class value can be read as "the <steps> element, defined within the "task" topic type module, is a specialization of the <ol> element defined in the "topic" topic type module. That is, it reflects the type hierarchy "steps is-a ol".

Given a DITA document with the @domains and@class attributes present the document is fully processable by any DITA-aware processor. In practice, however, the @domains and @class values are defined as attribute default values in some governing grammar and omitted from document instances. Thus, while the DITA standard does not require the use of grammars with conforming DITA documents, practical considerations effectively require grammars to enable population of the @domains and @class values, general document validation, authoring support, and so on.

The DITA standard defines specific coding requirements for the grammar types recognized by the DITA standard. These coding requirements ensure consistency across all conforming DITA vocabularies and thus facilitate interchange of vocabulary modules.

DITA vocabulary implementation involves three component types:

  • Vocabulary modules, which define element and attribute types. Vocabulary modules are either "structural", defining new map or topic types, or "domain", defining element types that may be used in any map or topic type as appropriate ("mix-in" elements). Every DITA document is either a map or a topic and therefore uses at least a single structural module. Vocabulary modules must be coded in such a way as to enable appropriate extension and constraint through separate modules. That is, it must be possible to modify the base content model and attribute list declarations as allowed by the DITA architecture without directly modifying the vocabulary module files themselves.

  • Constraint modules, which constrain content modules or attributes for elements. Constraints are implemented by redefining or overriding base content model and attribute list declarations.

  • Document type shells, which "integrate" vocabulary and constraint modules to implement specific DITA document types. Document type shells are logically just inclusion lists of the vocabulary and constraint modules used by the document type. In particular, the list of included modules should match the set of modules named by the document's @domains attribute.

The implementation challenge for DITA vocabulary, constraint, and document type shell authors is that neither DTD nor XSD are a particularly good match syntactically or semantically to DITA's requirements. DTDs require the use of parameter entities, which leads to convoluted and difficult-to-debug document type shells. XSD 1.0's limited and highly-constrained facilities for content model and group redefinition require various forms of indirection not inherent in the semantics of the DITA element types nor required in the DTD implementation of the equivalent vocabulary. The DITA user community, even highly-experienced practitioners, find implementing and maintaining both DTD and XSD vocabulary modules tedious and challenging.

While the issues with DTD mostly stem from the difficulty in debugging parameter entities and the large opportunity for error in constructing parameter entity declarations and references, the issues with XSD go deeper, relating to XSD's class model and restrictions on how groups may be redefined through the XSD redefine facility. In addition, the XSD 1.0 redefine facility is ambiguously specified such that conforming XSD processors may use either of two incompatible interpretations of how redefine should work. However DITA's use of XSD depends on the interpretation as implemented by the Xerces Java and C XSD parsers, making the use of XSD for DITA problematic in the general case. While the XSD 1.1 override facility appears to address the issues with redefine, until XSD 1.1 is implemented widely it cannot be mandated by the DITA standard.

Finally, the DITA Technical Committee, along with any other provider of DITA vocabularies, constraints, and document type shells faces the practical problem of maintaining both DTD and XSD versions of their vocabulary, which is challenging at best. An ideal solution would allow generation of either or both of the DTD and XSD versions from a single source. Through DITA version 1.2 the Technical Committee considered this too difficult to attempt from either the DTD or XSD as the authoritative source.

RELAX NG offers solutions to all these challenges by providing an XML-based syntax for document grammars that is a close equivalent to DTD syntax (avoiding the verbosity and conceptual overhead of XSD) while making the creation of document type shells much simpler, reducing them to essentially a simple inclusion list of modules. Because RELAX NG patterns may unilaterally augment other patterns, RELAX NG vocabulary and constraint modules become "self integrating", such that in most cases no additional work is required beyond simply creating a reference to the module from the document type shell.

Likewise, the relative simplicity of RELAX NG syntax coupled with an XML format make it practical to generate both DTD and XSD versions of the vocabulary from the RELAX NG.

DITA Modular Vocabulary Requirements

A DITA vocabulary module defines either a set of element types or a single global attribute. The grammar definition for a module must satisfy the following requirements:

  1. Provide the @domains attribute contribution for the module.

  2. Define content models with appropriate parameterization so that the following types of modifications may be made through the redefinition, extension, or override of those parameters:

    • Redefine the content model in its entirety

    • Allow additional element types where any given element type is allowed (integration of domain-provided specializations of base element types).

  3. Define attribute lists with appropriate parameterization so that the following types of modifications may be made through the redefinition, extension, or override of those parameters:

    • Redefine the attribute list in its entirety, for example, to omit specific attributes or set specific default values for specific attributes.

    • Extend the attribute list with global attributes defined in separate attribute domain modules.

  4. Provide common reusable content model fragments used to construct element-type-specific content models

Constraint modules use the parameterization facilities defined in modules to constrain element types. Thus any grammar facility must provide a way to include separate modules that can redefine or override parameters provided by modules.

Document type shells include ("integrate") the vocabulary and constraint modules in order to define a working document type. The document type shell must set the value of the @domains attribute as specified on the root element of the DITA document the document type shell governs.

Challenges Posed by DTD Syntax

As in all standard XML vocabularies intended for general use, the DITA DTDs make heavy use of parameter entities in order to enable configuration and constraint of vocabulary. The DITA DTD coding requirements specify the structure and naming rules for the parameter entities used to construct DITA element type and attribute list declarations.

The most important parameters are:

  • Element type name parameter entities for use in content models. For each element type defined in a vocabulary module there is a correponding parameter entity that expands to the element type name. These parameter entities are used in content models, e.g.:

    <!ENTITY % ph         "ph"            >


...

<!ENTITY % basic.ph 
  "%boolean; | 
   %cite; | 
   %keyword; | 
   %ph; | 
   %q; |
   %term; | 
   %tm; | 
   %xref; | 
   %state;
  "
>

  • "Domain integration" parameter entities, which are defined in domain modules and are used to override and extend the base values of element type name parameter entities in order to allow any specialization of the base type wherever the base type is allowed:

    <!ENTITY % hi-d-ph      
  "b | 
   i | 
   sup | 
   sub |
   tt | 
   u 
  "
>

    Each element domain module is represented by a pair of files, one named *.ent, the other named *.mod. The domain integration parameter entity is defined in the .ent file while the element-type-name parameter entities are defined in the .mod files. This use of two files for each module leads to one common error in constructing DTD document type shells: inclusion of the wrong file, leading to confusing and difficult-to-detect or understand errors.

  • Content model parameter entities, which define the content models for each element type. These parameter entities are always named "tagname.content". These parameter entities allow any element type to be constrained through separate constraint modules that override the base definition of the element's content model parameter entity, e.g.:

    <!ENTITY % body.content
                       "(%body.cnt; |
                         %bodydiv; | 
                         %example; | 
                         %section;)*"
>

As for all XML DTDs that make heavy use of parameter entities, one practical challenge is working out what the effective value of any given parameter entity is, as few, if any, DTD editing tools provide convenient ways to visualize expanded content models or follow references back to declarations.

Another practical problem is determining which declaration of a given parameter entity is the effective one when there are multiple declarations in multiple files. Error messages resulting from incorrect declarations are usually not very helpful in finding the actual cause of the error as the errors tend to be about incorrect content models resulting from the failure to resolve a particular entity or an error in the declaration of a redefinition of an existing entity. All complex, heavily-parameterized document types suffer from this problem and DITA's are as complicated as any. If DITA has an advantage in this area it is only that its coding patterns are both consistent and mandated by the standard, making it slightly easier to detect typing errors.

DTD-syntax document type shells in DITA require multiple external parameter entity references, in specific orders, for each structural and domain module as well as declaration of the domain integration parameter entities, again in the right order, as well as inclusion of any constraint modules used in the shell.

The need for multiple external parameter entities for each module adds complexity and opportunities for error. For example, it is easy to accidentally include the wrong file, for example, including a .mod file instead of the required .ent file. This type of error can be hard to find because it doesn't necessarily make the resulting DTD invalid but may make it incorrect because the order of parameter entity declarations is not correct, causing subsequent redefinitions to have no effect.

Likewise, it is easy to make typos in the domain entity integration parameter entity declarations, for example, omitting a "|" separator or simply misspelling the name of a domain-defined parameter entity. Or simply forgetting to include a reference to domain's integration parameter entity. None of these errors is easy to check programmatically and none are reported in a way that makes the real error obvious. Even experienced DITA DTD authors can find it difficult to diagnose and resolve these types of errors.

While the DITA standard defines and documents the coding pattern for document type shells such that anyone should be able to simply follow the pattern in order to successfully create their own custom shell, in practice few people even attempt it and fewer succeed because of the inherent challenges of DTD syntax.

Challenges Posed by XML Schema

The XML Schema Document (XSD) facility presents several challenges for use with DITA vocabularies:

  • The requirement to use the XSD redefine facility

  • Limits on redefine and the resulting need for intermediate groups and, in some cases, intermediate files

  • Declarations for integrating domain-provided element types into base content models

  • The verbosity of the markup generally stemming from the XSD class and element type distinction, which does not correspond to DITA's class concept.

XSD Redefine Facility

The XSD redefine facility is defined ambiguously in the XSD recommendation such that conforming XSD processors may use either of two incompatible interpretations of redefinition, only one of which works for DITA. This means only some XSD processors can be used to validate XSD-based DITA documents. Fortunately, two of those processors are the Xerces Java and C-language XSD processors, making the required XSD processing available in all Java environments and many non-Java environments. In XSD 1.1 the Redefine facility is deprecated:

4.2.4 Including modified component definitions (<redefine>)

Note: The redefinition feature described in the remainder of this section is ·deprecated· and may be removed from future versions of this specification. Schema authors are encouraged to avoid its use in cases where interoperability or compatibility with later versions of this specification are important.

The primary issue appears to be with systems that cache schemas. If two documents use two different top-level schemas, each of which redefines the same base model from a third schema, it is possible that a system may cache the redefine as defined by the first top-level schema and use it when processing against the second top-level schema. Any system that does this will reliably fail with typical DITA documents where there is a mix of topic types, each of which necessarily provides different redefines for the same base models. It is not clear if any XSD processors actually have this caching behavior.

The XSD 1.1 override facility, which was driven in part by DITA requirements, appears to provide the override semantics DITA requires, but as XSD 1.1 is not universally implemented it is difficult to justify requiring the use of XSD 1.1. However, because the DITA XSDs are generated starting with DITA 1.3, it would be possible to generate XSDs that use override in place of redefine if the DITA community requires them. As of May 2014 it appears that the only XSD 1.1 implementations are Xerces-J and Saxon EE, which is not sufficient to support the full DITA community.

The challenge presented by the XSD Redefine facility is its "particle preservation" requirement, defined as follows:

The definitions within the <redefine> element itself are restricted to be redefinitions of components from the <redefine>d schema document, in terms of themselves. That is,

  • Type definitions must use themselves as their base type definition;

  • Attribute group definitions and model group definitions must be supersets or subsets of their original definitions, either by including exactly one reference to themselves or by containing only (possibly restricted) components which appear in a corresponding way in their <redefine>d selves.[1]

This means that any redefinition of a model must reflect each of the particles in the original model. For choice groups this is not a problem: any choice group is a valid restriction, including an empty group. But for sequence groups it is a serious problem, in that you cannot simply omit items from the sequence as part of the redefinition.

For example, given this base schema:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
  xmlns:s1="urn:example.com:schema-01"
  targetNamespace="urn:example.com:schema-01"
  >
  
  <xs:complexType name="root.type">
    <xs:sequence>
      <xs:element ref="s1:A"/>
      <xs:element ref="s1:B"/>
      <xs:element ref="s1:C"/>
    </xs:sequence>
  </xs:complexType>
  
  <xs:element name="root" type="s1:root.type">
  </xs:element>
  
  <xs:element name="A" >    
  </xs:element>
  <xs:element name="B" >    
  </xs:element>
  <xs:element name="C" >    
  </xs:element>
  
</xs:schema>

This redefinition is invalid:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
   xmlns:s1="urn:example.com:schema-01"
  targetNamespace="urn:example.com:schema-01"
  >
  
  <!-- Redefine of redefined schema 01 -->
  <xs:redefine schemaLocation="redefined-schema-01.xsd">
    <xs:complexType name="root.type">
      <xs:complexContent>
        <xs:restriction base="s1:root.type">
          <xs:sequence>
            <xs:element ref="s1:A"/>
            <xs:element ref="s1:C"/>
          </xs:sequence>
        </xs:restriction>
      </xs:complexContent>
    </xs:complexType>
  </xs:redefine>
</xs:schema>

Because the "B" particle from the origin sequence is not accounted for in the redefined sequence.

Likewise, this attempt to make "B" disallowed is also invalid:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
   xmlns:s1="urn:example.com:schema-01"
  targetNamespace="urn:example.com:schema-01"
  >
  
  <!-- Redefine of redefined schema 01 -->
  <xs:redefine schemaLocation="redefined-schema-01.xsd">
    <xs:complexType name="root.type">
      <xs:complexContent>
        <xs:restriction base="s1:root.type">
          <xs:sequence>
            <xs:element ref="s1:A"/>
            <xs:element ref="s1:B" maxOccurs="0" minOccurs="0"/>
            <xs:element ref="s1:C"/>
          </xs:sequence>
        </xs:restriction>
      </xs:complexContent>
    </xs:complexType>
  </xs:redefine>
</xs:schema>

In both cases, the Xerces J parser reports that there is not a complete functional mapping between the particles in the original model and the redefined model.

The solution is to refactor the sequence into a sequence of named groups. This allows the groups to be referenced from a redefinition, satisfying the requirement that a redefinition must include the thing being redefined:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" 
   xmlns:s1="urn:example.com:schema-01" targetNamespace="urn:example.com:schema-01">

  <xs:group name="A.grp">
    <xs:sequence>
      <xs:choice>
        <xs:element ref="s1:A"/>
      </xs:choice>
    </xs:sequence>
  </xs:group>
  <xs:group name="B.grp">
    <xs:sequence>
      <xs:choice>
        <xs:element ref="s1:B"/>
      </xs:choice>
    </xs:sequence>
  </xs:group>
  <xs:group name="C.grp">
    <xs:sequence>
      <xs:choice>
        <xs:element ref="s1:C"/>
      </xs:choice>
    </xs:sequence>
  </xs:group>

  <xs:complexType name="root.type">
    <xs:sequence>
      <xs:group ref="s1:A.grp"/>
      <xs:group ref="s1:B.grp"/>
      <xs:group ref="s1:C.grp"/>
    </xs:sequence>
  </xs:complexType>

  <xs:element name="root" type="s1:root.type"> </xs:element>

  <xs:element name="A"> </xs:element>
  <xs:element name="B"> </xs:element>
  <xs:element name="C"> </xs:element>

</xs:schema>

Note that the pattern of xs:sequence/xs:choice is required in order to control occurrence of the choice (e.g., to make the choice optional or repeatable).

The redefinition can then look like this:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
   xmlns:s1="urn:example.com:schema-01"
  targetNamespace="urn:example.com:schema-01"
  >
  
  <!-- Redefine of redefined schema 01 -->
  <xs:redefine schemaLocation="redefined-schema-02.xsd">
     <xs:group name="B.grp">
       <xs:sequence>         
         <xs:choice minOccurs="0" maxOccurs="0">
           <xs:group ref="s1:B.grp"/>
         </xs:choice>
       </xs:sequence>
     </xs:group>
  </xs:redefine>
</xs:schema>

In this case, making the "B" element effectively disallowed while maintaining the required particle-to-particle association.

This need for additional named groups for sequences within content models makes translating content models as normally formulated in DTD or RELAX NG grammars into XSD tedious at best and challenging at worst. It is not clear if there is a general algorithm for translating arbitrary content models with sequences into the necessary XSD groups in order to then enable arbitrary constraint via redefinition. In the case of the DITA XSDs, at least through DITA 1.2, this type of group refactoring has been done only as needed to support constraints defined by the DITA Technical Committee. This means that there are many element types that cannot be constrained using the XSD modules as provided by the DITA Technical Committee. This issue is explored in more detail below.

DITA 1.2 Group Design and XSD

In DITA every base element type may be extended through the creation of "domain" element specializations such that anywhere the base element type is allowed the domain-defined specializations are also allowed.

This is implemented in all grammar languages by defining the base content models using references to some type of parameter that initially resolves to just the base element type but that may be redefined to expand to the base type and its domain-provided specializations. Document type shells may also omit the base type or any specialized type, effectively imposing a constraint on the base element type as part of the domain integration. For example, given a domain that specializes the general inline element type <ph>, you might allow only the specialized types and omit <ph>. The naming pattern used in all grammar languages gives the parameter the same name as the element type, e.g., "%ph" in DTDs, <define name="ph"> in RELAX NG, and <xs:group name="ph"> in XSD.

For XSD this pattern means that there is one element-type-specific group for each element type defined in each vocabulary module. The base definition of each group simply allows the corresponding element:

<xs:group name="author">
  <xs:sequence>
    <xs:choice>
      <xs:element ref="author"/>
    </xs:choice>
  </xs:sequence>
</xs:group>

For example, a domain could extend the <author> element by defining two specializations of <author>, say <editor> and <primary-author>. Within document type shell XSD documents that integrate this domain the "author" group is redefined like so:

  <xs:redefine schemaLocation="urn:oasis:names:tc:dita:xsd:metaDeclGrp.xsd">
    <xs:group name="author">
      <xs:choice>
        <xs:group ref="author"/>
        <xs:group ref="jrnlmeta-d-author" />
      </xs:choice>
    </xs:group>
  </xs:redefine>

The group "jrnlmeta-d-author" is defined like so:

  <xs:group name="jrnlmeta-d-author">
    <xs:choice>
      <xs:element ref="editor" />
      <xs:element ref="primary-author" />
    </xs:choice>
  </xs:group>

This redefinition satisfies the particle preservation requirement of the redefine feature because the original particle "author" is maintained with additional new particles added within a choice group.

Note that each element-type-specific group must itself require its contained choice group. If this were not the case it would be impossible to define content models that required elements. Thus the element-type-specific groups themselves cannot be used to work around the redefinition particle preservation requirement.

Working Around Redefine Limitations: Indirection in Sequence Groups

XSD redefinition is applied at the group level such that every particle in the original group must be represented in the redefined group. When the group defines a sequence it is not possible to redefine the sequence in order to omit tokens.

For example, the DITA <prolog> element uses this sequence group as its abstract content model: 

 (author*, 
  source?, 
  publisher?,
  copyright*, 
  critdates?,
  permissions?, 
  metadata*, 
  resourceid*,
  (data.elements.incl | 
   foreign.unknown.incl)*)

Each element is optional. Per the rule for DITA constraints you can omit any optional element. This means that any restriction of the content model that omits one or more particles is a conforming constraint.

If the XSD implementation of the content model is defined as a single group the resulting group is will not allow constraints that omit optional types:

<xs:sequence>
  <xs:group ref="author" minOccurs="0" maxOccurs="unbounded"/>
  <xs:group ref="source" minOccurs="0"/>
  <xs:group ref="publisher" minOccurs="0"/>
  <xs:group ref="copyright" minOccurs="0" maxOccurs="unbounded"/>
  <xs:group ref="critdates" minOccurs="0"/>
  <xs:group ref="permissions" minOccurs="0"/>
  <xs:group ref="metadata" minOccurs="0" maxOccurs="unbounded"/>
  <xs:group ref="resourceid" minOccurs="0" maxOccurs="unbounded"/>
  <xs:choice minOccurs="0" maxOccurs="unbounded">
    <xs:group ref="data.elements.incl"/>
    <xs:group ref="foreign.unknown.incl"/>
  </xs:choice>
</xs:sequence>

The general solution is to add another layer of grouping such that for each contiguous sequence of omittable elements there is a group whose content is an emptiable choice or sequence.

For example, the base content model for the DITA <taskbody> element is:

((prereq | 
  context |
  section)*,
 (steps | 
  steps-unordered |
  steps-informal)?, 
 result?, 
 example*, 
 postreq*)

The DITA standard defines a constraint on this content model, the "strict task" constraint, which omits the <section> and <steps-informal> elements.

If the XSD implementation of this content model were simply the equivalent sequence group, it would not be constrainable because the particle preservation rule does not allow omission of the <section> or <steps-informal> groups.

One solution is to insert three groups: one for the elements before <steps>, one for the <steps> optional choice group, and one for the elements after <steps>:

  <xs:group name="taskbody.content">
    <xs:sequence>
      <xs:group ref="taskPreStep"/>
      <xs:group ref="taskStep"/>
      <xs:group ref="taskPostStep"/>
    </xs:sequence>
  </xs:group>

  <xs:group name="taskPreStep">
    <xs:sequence>
      <xs:choice minOccurs="0" maxOccurs="unbounded">
        <xs:group ref="context" minOccurs="0"/>
        <xs:group ref="prereq"  minOccurs="0"/>
        <xs:group ref="section" minOccurs="0"/>
      </xs:choice>
    </xs:sequence>
  </xs:group>

  <xs:group name="taskPostStep">
    <xs:sequence>
      <xs:group ref="result"  minOccurs="0"/>
      <xs:group ref="example"  minOccurs="0" maxOccurs="unbounded"/>
      <xs:group ref="postreq"  minOccurs="0" maxOccurs="unbounded"/>  
    </xs:sequence>
  </xs:group>
  
  <xs:group name="taskStep">
    <xs:sequence>
      <xs:choice minOccurs="0" maxOccurs="1">
        <xs:group ref="steps" />
        <xs:group ref="steps-unordered" />
        <xs:group ref="steps-informal" />
      </xs:choice>
    </xs:sequence>
  </xs:group>

This extra level of grouping enables the constraint for <taskbody>. However, no other sequence group content model in the DITA 1.2 XSD schemas provides a comparable set of emptiable groups, meaning that more than 150 element types in the DITA 1.2 base vocabulary are not constrainable without first modifying the OASIS-provided XSDs, which violates the basic DITA principle that all copies of a given vocabulary module are identical.

This presents an implementation challenge for the XSD generation process: is there a general algorithm that will generate constrainable model groups that allow any valid constraint (in particular, allowing removal of optional elements?

Declarations for Integrating Domain Element Types in XSD Document Type Shells

Domain-provided specialized element types are integrated through redefinitions specified in document type shells. The markup itself is not particularly complicated, but it must be specified in every document type shell that integrates a given domain. Domain-provided elements types are "mix-in" elements, meaning that the domain-provided elements must be allowed wherever the element they specialize is allowed. Conceptually this is equivalent to an XSD substitution group. However, it appears that XSD substitution groups do not work reliably for this and thus are not used by the DITA XSDs.

Because the integration of each domain requires explicit redefinition in the XSD document type shell, XSD domains are not self integrating.

The names of the groups to be referenced in the redefined groups are determined by the short name of each integrated domain. The DITA XSD coding requirements specify that the domain integration groups must be named such that, given the domain short name, you can reliably construct the domain integration group name.

A typical document type shell is the OASIS-provided concept topic type shell:

  <xs:redefine schemaLocation="urn:oasis:names:tc:dita:xsd:commonElementGrp.xsd:1.2">
    <xs:group name="keyword">
      <xs:choice>
        <xs:group ref="keyword"/>
        <xs:group ref="pr-d-keyword" />
        <xs:group ref="ui-d-keyword" />
        <xs:group ref="sw-d-keyword" />
      </xs:choice>
    </xs:group>

    <xs:group name="ph">
      <xs:choice>
        <xs:group ref="ph"/>
        <xs:group ref="pr-d-ph" />
        <xs:group ref="ui-d-ph" />
        <xs:group ref="hi-d-ph" />
        <xs:group ref="sw-d-ph" />
      </xs:choice>
    </xs:group>

    <xs:group name="pre">
      <xs:choice>
        <xs:group ref="pre"/>
        <xs:group ref="pr-d-pre" />
        <xs:group ref="ui-d-pre" />
        <xs:group ref="sw-d-pre" />
      </xs:choice>
    </xs:group>

    <xs:group name="dl">
      <xs:choice>
        <xs:group ref="dl"/>
        <xs:group ref="pr-d-dl"/>
      </xs:choice >
    </xs:group >

    <xs:group name="fig">
      <xs:choice>
        <xs:group ref="fig"/>
        <xs:group ref="pr-d-fig"/>
        <xs:group ref="ut-d-fig" />
      </xs:choice>
    </xs:group >
    
    <xs:group name="index-base">
      <xs:choice>
        <xs:group ref="index-base"/>
        <xs:group ref="indexing-d-index-base"/>
      </xs:choice>
    </xs:group >
      
    <xs:group name="note">
      <xs:choice>
        <xs:group ref="note"/>
        <xs:group ref="hazard-d-note"/>
      </xs:choice>
    </xs:group >
    
    <xs:group name="term">
      <xs:choice>
        <xs:group ref="term"/>
        <xs:group ref="abbrev-d-term"/>
      </xs:choice>
    </xs:group >

  </xs:redefine>

How RELAX NG Addresses DITA Requirements

RELAX NG provides an almost ideal match of syntax and semantics to DITA vocabulary architecture requirements:

  • The RELAX NG pattern approach is a good match to DTD semantics and thus to DITA semantics.

  • The ability to extend patterns unilaterally makes modules self-integrating, reducing the complexity of document type shells.

  • RELAX NG's XML syntax plus semantic match to DTDs makes generating conforming DITA DTDs practical.

  • RELAX NG's XML syntax makes generating XSDs practical, if challenging due to requirements for XSDs.

The use of RELAX NG for DITA does require two things not in the base RELAX NG standard:

  • Attribute default values. These are provided for by the companion RELAX NG DTD Compatibility standard. One of us (Bina) implemented support for attribute defaults as an enhancement to the open-source JING library, making RELAX NGs directly usable for DITA document processing in any Java processing environment.

  • DITA-specific metadata for vocabulary and constraint modules and document type shells. DITA-specific metadata is required in order to enable generation of DTD and XSD versions of the RELAX NG document type shells and modules, such as the public IDs to use for specific generated modules, the module types, and module short names, as well as DITA-specific properties for individual element types, such as a long name and the initial DITA versions for patterns and pattern components.

The one tedious aspect of RELAX NG that adds complexity to the DITA document type shells is the need to have special declarations in the shell for elements that define @id attributes of type "ID".

In DITA most elements define the @id attribute as NMTOKEN rather than as an XML ID. However, some element types, in particular, <topic> and all specializations of <topic>, use XML IDs. RELAX NG's rules for "any" patterns do not allow two different element types to have different datatypes for the same attribute. DITA document type shells must work around this by redefining the "any" patterns to explicitly exclude those element types or foreign namespaces that define @id attributes as type ID. However, for most shells this redefinition is the same except for the tagnames of the topic types defined or included in the shell.

The only other component of document type shells that must be set manually is the value of the @domains attribute for the root topic or map element defined by the shell as RELAX NG provides no way to dynamically construct the pattern for attribute defaults. However, each vocabulary and constraint module declares its @domains contribution in DITA-specific metadata. The DITA Technical Committee provides a simple XQuery script that updates the value of the @domains attribute to reflect the modules actually included, removing the need to manually set and maintain the @domains value.

RELAX NG is a Good Match to DTDs

The RELAX NG pattern approach, as opposed to the XSD type/element approach, provides a close semantic and syntactic match to XML DTDs, making it relatively easy for DTD authors to transition to RELAX NG as their grammar language. There is an almost one-to-one correspondence between DTD syntactic constructs and RELAX NG grammar components. This makes it relatively easy to translate existing DTD-syntax vocabulary modules into the equivalent RELAX NG grammars. It also allows the DITA-defined RELAX NG coding requirements to be as close to the DTD coding requirements as possible, maximizing the applicability of existing DTD coding knowledge to DITA RELAX NG grammars.

The RELAX NG compact syntax is also syntactically similar to DTD syntax. The compact syntax is less convenient for DITA vocabularies because DITA grammars do require use of non-RELAX NG elements and attributes but it is still usable as RELAX NG provides ways of encoding foreign elements and attributes in the compact syntax. The DITA Technical Committee generates RELAX NG compact syntax versions of all the TC-defined DITA vocabulary.

Modules are Self Integrating

The most dramatic advantage of RELAX NG over both DTD and XSD is the ability for one pattern to unilaterally extend another pattern. This allows DITA domain modules to automatically extend the patterns for the base elements those domains specialize.

As for DTDs and XSDs, each element type is represented by a pattern whose name is the same as the element type:

  <define name="ph">
    <ref name="ph.element"/>
  </define>

The domain specialization of a given element simply extends the element-type pattern to add the specializations provided by the domain:

    <define name="hi-d-ph">
      <choice>
        <ref name="b.element"/>
        <ref name="i.element"/>
        <ref name="line-through.element" dita:since="1.3"/>
        <ref name="overline.element" dita:since="1.3"/>
        <ref name="sup.element"/>
        <ref name="sub.element"/>
        <ref name="tt.element"/>
        <ref name="u.element"/>
      </choice>
    </define>

    <define name="ph" combine="choice">
      <ref name="hi-d-ph"/>
    </define>

The combine="choice" on the domain-specific definition of the "ph" pattern in this example makes the effective value of the "ph" pattern the base tokens plus all the tokens defined in the "hi-d-ph" pattern. This corresponds directly to the domain integration parameter entities in DTDs and domain integration redefines in XSDs, but requires no additional work in document type shells.

The document type shell can then be a simple set of module inclusions. This eliminates almost all of the complexity and opportunity for error of DTD and XSD document type shells. In short, it makes creating DITA document type shells about as easy as it could possibly be.

DTD and XSD Generation

While RELAX NG has many advantages for DITA vocabulary authors the DITA community still requires DTDs and, to a lesser extent, XSDs. Almost all DITA-aware tools are DTD-aware and a few are XSD-only. Few support use of RELAX NG today. Thus the DITA Technical Committee must continue to provide DTD and XSD versions of all TC-defined vocabulary. In addition, these DTDs and XSDs must themselves conform to the DITA coding requirements for DTDs.

One of us (Kimber) has implemented XSLT-based transforms to generate conforming DTD, XSD, and RELAX NG compact syntax modules and shells from the RELAX NG XML syntax modules. These transforms allow the DITA Technical Committee to make the RELAX NG vocabulary the authoritative master from which all other forms are generated, significantly reducing the cost of maintaining multiple grammar formats for the DITA vocabulary.

Conclusions and Future Work

As an alternative to DTDs, RELAX NG has proven to be superior to XSD in almost every way. It is provides a good semantic and syntactic match to DTDs. It's rules for extending and overriding patterns work well for DITA's specialization and constraint features. It's XML syntax makes implementing generation of other formats practical using normal XML tools (XSLT, XQuery). The ability to unilaterally extend patterns makes RELAX NG document type shells about as easy as it can be. Even if used only as a source for then generating the DTDs and XSDs used with DITA processing tools, RELAX NG offers significant benefits to DITA document type shell and module authors.

Through this experience we have identified potential improvements to RELAX NG that would make its use even simpler:

  • Relax the constraints on attribute types within "any" patterns.

  • Provide a mechanism for dynamically constructing attribute value defaults

With these two refinements, DITA modules would be complete self-integrating, making DITA document type shells truly just inclusion lists of modules.

The question of generation of constrainable XSD content models remain open. More thought and experimentation is required but at the time of writing it is not clear that a general algorithm for translating sequence groups to the necessary XSD groups is either possible or desirable. In addition, it is not clear that there is sufficient use of XSDs within the DITA community with non-TC-provided constraints to justify the effort of trying to generate constrainable XSDs. One piece of evidence that this is the case is the fact that the issue with the general lack of constrainability was not discovered until we started implementing the XSD generation process, meaning that no DITA user reported any issue with XSD constraints between the time DITA 1.1 was released and now. Either nobody tried to implement constraints of sequence groups using XSDs in that time or anyone who did simply modified their TC-provided XSD files locally and never reported the issue to the DITA Technical Committee or to the DITA community at large.

At the time of writing the decision of the DITA Technical Committee is to generate the DITA 1.3 XSDs using the same content model patterns as for DITA 1.2, treating those content models that require constraint (e.g., taskbody) as special cases within the XSD generation transform. Because the XSDs are generated the Technical Committee can add additional special cases as required, replace the use of Redefine with Override, or improve the general model generation algorithm.

References

[dita-std] Darwin Information Typing Architecture, OASIS Open. http://docs.oasis-open.org/dita/v1.2/os/spechttp://docs.oasis-open.org/dita/v1.2/os/spec

[xml-std] Extensible Markup Language, W3C. http://www.w3.org/TR/xml11/

[relaxng-std] RELAX NG, OASIS Open. http://relaxng.org/spec-20011203.html

[rng-dtd-compat-std] RELAX NG DTD Compatibility, OASIS Open. http://relaxng.org/compatibility-20011203.html

[xsd-std1.1] XML Schema Part 1: Structures Second Edition, W3C. http://www.w3.org/TR/2004/REC-xmlschema-1-20041028/

W3C XML Schema Definition Language (XSD) 1.1 Part 1: Structures, W3C. http://www.w3.org/TR/xmlschema11-1


[1] XML Schema Part 1: Structures Second Edition, clause 4.2.2 Including modified component definitions

Author's keywords for this paper:
DITA; document grammar; DTD; XSD; RELAX NG; XML schema document; vocabulary architecture; Darwin Information Typing Architecture (DITA)

Kimber Eliot

Owner

Contrext, LLC

Eliot has been doing generalized markup longer than most humans have been alive. Eliot's most recent professional focus is on the application of the DITA standard to the requirements of professional publishers. Eliot is a founding member of the OASIS DITA Technical Committee and active contributor to the DITA standard. Eliot maintains the DITA For Publishers open source project. When not wrangling tags to his will, Eliot trains in Aikido and skateboards when he can. Eliot lives in Austin, Texas.

Bina George

SyncroSoft

George is founder of SyncroSoft, makers of the oXygenXML product. George is a mathematician by training. George lives and works in Criova, Romania. When not contributing to open-source projects, George enjoys travel with his family and taking selfies of himself working in exotic locations.