Balisage Paper: Structural Metadata and the Social Limitation of Interoperability: A Sociotechnical View of XML and Digital Library Standards Development

Introduction

Ten years after its endorsement by the World Wide Web Consortium, XML has achieved a high degree of adoption within numerous, disparate communities and in a vast range of application domains, from standards for electronic filing of federal income tax (Internal Revenue Service, 2007) to user interface design (Goodger et al., 2001). The digital library community has been an active and early adopter of XML, for use in structuring both content and metadata. The reasons for this rapid uptake of XML within the digital library community are familiar to anyone with experience in the world of markup languages:

For all of these reasons, XML-based content standards such as the Text Encoding Initiative (TEI) have seen wide adoption within the library community, and librarians have been actively engaged in the development of a number of XML-based metadata standards, including Encoded Archival Description (EAD), Metadata Object Description Schema (MODS), Metadata Authority Description Schema (MADS), Metadata Encoding and Transmission Standard (METS), Metadata for Images in XML (MIX), MPEG-21 Digital Item Declaration Language (DIDL), Open Archives Initiative Object Reuse and Exchange (OAI-ORE), Preservation Metadata Implementation Strategies (PREMIS) and many others. XML is now widely used throughout the research library world, and is a fundamental part of the infrastructure developed within the digital library community over the past decade.

Despite its success, however, XML has not lived up to many librarians' expectations within one area, that of interoperability. Efforts to exchange information employing XML incarnations of descriptive metadata standards such as Dublin Core have fallen prey to a number of encoding and semantic inconsistencies (Shreeves et al., 2005). Perhaps more surprising is the failure of XML to ensure interoperability at a syntactic level. Digital library developers have expected that shared use of an XML standard for structuring of content and metadata (what is commonly called "structural metadata" within the digital library community) would ensure content interoperability and provide a clean division between content and higher level tools and services designed to work with standardized encodings of that content (Hurley et al., 1999). In practice, this goal has proved extraordinarily elusive. Experiments conducted by participants in the Library of Congress National Digital Infrastructure for Preservation Program (NDIIPP)^[1] to test the exchange of digital objects between repositories failed even when participants were using the same XML-based encoding format and producing valid XML instances to exchange (DiLauro et al., 2005, Shirky, 2005).

While some of the failures experienced by the Library of Congress NDIIPP tests were the result of incompatible repository infrastructure, others resulted from mismatched expectations regarding the appropriate use of METS, the XML format employed for the test. DiLauro et al., discussing Johns Hopkins University's experience in the NDIIPP tests, state:

As DiLauro et al. allude, a critical difficulty for achieving interoperability using structural metadata standards such as METS is the level of flexibility they enable in structuring a description of an object. As (Nelson et al., 2005) note in their discussion of using the MPEG-21 Digital Item Declaration Language during the NDIIPP test, it is possible to map a single object into multiple different encodings using MPEG-21, depending on the level of granularity you wish to employ in the description. The same is true of METS and other, similar information packaging standards. They each provide a structural grammar for the encoding of complex digital objects, and document authors can and do find a variety of different ways to talk about the structure of a single object using one of those grammars.

To date, the digital library community has treated these interoperability issues surrounding structural metadata standards as a technical problem demanding a technical solution. Most efforts to solve the interoperability problems around structural metadata have focused on the use of a profiling mechanism to further constrain the creation of instance documents conforming with the XML schema, sometimes in conjunction with a validation mechanism (such as Schematron) to test instance documents conformance with the additional requirements established in the profile (Littman, 2006, Keith, 2005). However, while these mechanisms may be successful in insuring interoperability within a narrowly defined local context, they are not in themselves any guarantee of interoperability at the scale envisioned by digital library projects such as Aquifer (Kott et al., 2006), which hope to promote the ready exchange and interoperability of digital library content among a multitude of institutions. The official METS profile registry already contains a variety of mutually incompatible profiles for similar types of objects.^[2] Allowing the specification of local profiles may formalize the problem, but it does not solve it.

If we are to deal with the issues of interoperability that continually manifest themselves in the realm of structural metadata standards for digital libraries, I believe we need to cease viewing this purely as a technical problem, and acknowledge that it is the result of the interplay of a variety of technical and social factors. The XML standards for structural metadata employed by the digital library community represent cases of sociotechnical systems, and only when we have analyzed the social, as well as the technical, components of these systems will we be in a position to suggest how they may be optimized to achieve the goals of their users.

A Sociotechnical View of XML

All of the existing and developing standards for structural metadata^[3] within the digital library community are XML-based. Any sociotechnical examination of these standards therefore must start with at least some consideration of XML itself. (Akrich, 1992) argues that

Our question with respect to XML, then, is what worldview have XML's authors inscribed within it and what influence has that had on XML's uptake and use within the digital library community.

We can learn a great deal about the viewpoints of a particular technology's designers from the documents they author which either help define and implement the goals for the technology (e.g., use cases, user needs analysis, specification documents), or which attempt to explain or promote new technology to potential users. If we look at the original XML Recommendation, we can find a relatively clear set of goals for the technology in the "Origin and Goals" section:

These goals convey some of the world view that XML's designer brought to bear in creating the technology. It is clear that they saw XML as a transmission format for communications (hence the requirement that it be usable over the Internet). They believed that XML's success was contingent upon it being flexible enough to "support a wide variety of applications." They also clearly believed that for XML to succeed it must be easy to use, but they also recognized that the meaning of 'ease of use' is contingent upon the use one might make of the technology. Ease of use for a document author ("XML documents shall be easy to create," "XML documents should be human-legible and reasonably clear") is a good deal different from ease of use for a software developer ("It shall be easy to write programs which process XML Documents," "The design of XML shall be formal and concise").

This relatively small set of goals for the XML language was further elaborated upon by members of the original W3C XML Working Group in a variety of papers they authored to introduce and clarify XML to its potential user community. (Bosak, 1998) further defines the goals of XML as supporting the user needs of "extensibility, to define new tags as needed," "structure, to model data to any level of complexity," "validation, to check data for structural correctness," "media independence, to publish content in multiple formats," and "vendor and platform independence, to process any conforming document using standard commercial software or even simple text tools." The benefits adhering to XML's providing a standardized format include "complete interoperability of both content and style across applications and platforms; freedom of content creators from vendor control of production tools; freedom of users to choose their own views into content; easy construction of powerful tools for manipulating content on a large scale; a level playing field for independent software developers; and true international publishing across all media." Emancipatory language is invoked repeatedly here through the use of the terms 'freedom' and 'independence,' particularly affording users the freedom "to define new tags" and in so doing "choose their own views into content." A key benefit to XML in this world view is the freedom it provides users to define their own structure for documents and data, using their own semantics, and to escape restrictions that software vendors (through their own inscriptions in their own products) might wish to impose on their users. Other articles by members of the XML Working Group (see, for example, Bosak, 1997 and Bosak and Bray, 1999) reiterate a vision of XML as a technology allowing users to define their own structures while simultaneously supporting interoperability of documents and data on a global scale.

The XML 1.0 Recommendation can be seen as bearing the inscription of its designers' ideological stance towards appropriate mechanisms for data and document structuring as well as appropriate relationships between document creators and software and platform vendors. The effort to promote a meta-language over any specific markup language, the adoption of Unicode as a basic character set, and the elimination of SGML features such as CONCUR, OMITTAG and SUBDOC are some of the technological means through which XML's designers sought to normalize and reify a particular set of social and technological relationships. Nor did this process stop with the release of the XML 1.0 recommendation in 1998. The period between February 1998 and October 2001 saw the development and release of a plethora of additional XML specifications, including XML Namespaces, XSLT, XPath, XML Schema, XLink/XBase, XML Information Set and XSL-FO, as well as a variety of XML software tools including parsers, editors and stylesheet engines. All of these various technological objects presented their own opportunities for their designers to further refine the ideological inscription carried within the XML 1.0 Recommendation. One of these objects in particular, the Namespaces in XML Recommendation, deserves further examination due to its significant impact on structural metadata standards developed by the digital library community.

(World Wide Web Consortium, 1999a) provides the following justification for the introduction of a formal namespace mechanism into XML:

A strong motivating force for the Namespaces in XML recommendation, then, was a desire to promote modularity in the design and implementation of markup languages. Interoperability was also cited by the World Wide Web Consortium in the introduction of Namespaces in XML (World Wide Web Consortium, 1999b). Fundamentally, the authors of the Namespaces in XML recommendation wanted to simplify XML document authors' lives by insuring that they did not need to reinvent markup languages which already existed, and that they could readily mix elements and attributes conforming to disparate schemas within a single document instance without worrying about collisions between element and attribute names.

While it was not widely recognized at the time, the combination of the XML namespaces and the XML Schema recommendations provided the digital library community with the precise combination of tools that it had been seeking for some two years, since the publishing of the Warwick Framework document (Lagoze, Lynch and Daniel, 1996). The Warwick Framework described a container architecture that would allow a digital library practitioner to bind together and encapsulate different forms of metadata describing a particular object, utilizing a modular system in which different forms of metadata could be plugged in as needed into the larger framework. The description of the Warwick Framework in the original proposal document bears at least some similarity to the list of benefits for XML itself included above:

While initial discussion of specific implementations of the Warwick Framework focused on RDF (see Daniel and Lagoze, 1997), the release of the XML Namespace and Schema recommendations seemed to provide the exact combination of flexibility, modularity, extensibility and interoperability that the digital library community needed for such a framework.

Structural Metadata Standards & the Digital Library Community

In the same year that the XML 1.0 Recommendation was released, a research project undertaken by U.C. Berkeley Library, Cornell University Library, New York Public Library, Penn State University Library and Stanford University Library sought "to develop best practices for the encoding of intellectual, structural and administrative data about primary resources housed in research libraries" (U.C. Berkeley Library, 1997). The project, known as the Making of America II (MOA2), sought to test its notions of "best practices" by developing a test bed repository of digital library materials that would be created by the participating institutions; the content and metadata for these digital library objects were to be transmitted to the central repository, housed at the U.C. Berkeley Library, using a transmission format to be developed by the project.

Given the participants' prior experience using SGML standards such as TEI and EAD, and the recent release of the XML 1.0 Recommendation, it was decided that the transmission format should be developed using XML. This format, known as the MOA2 DTD^[4], was intended to convey descriptive metadata and administrative metadata about a digital library object, as well as providing a structural metadata framework which allowed MOA2 document authors to describe the logical or physical structure of a work (e.g., 'this is a book with chapters' or 'this is a bound volume with pages'), to associate portions of that structure with digital files representing the content of each portion, and to associate descriptive and administrative metadata with either segments of the work, or with the digital content files themselves. As the transmission format needed to support a variety of work genres (including still photographs, journals, ledgers and diaries), the elements for describing the structure of the object were highly abstract. Drawing upon previous work done by the Text Encoding Initiative, the MOA2 DTD represented the structure of any digital library object as a series of nested <div> elements. Child <fptr> elements underneath a <div> associated the <div> with digital content files, and a variety of XML ID/IDREF attributes were used to link <div> elements and content files with descriptive and administrative metadata.

While the MOA2 DTD was reasonably successful, experiments with the use of the format outside the original research project revealed several deficiencies, including a lack of support for time-based media, and what was seen by members of the digital library community as an excessively restrictive model with respect to the descriptive and administrative metadata allowed within a MOA2 document. In this latter regard, the MOA2 DTD was viewed as not living up to the promise of modular design and flexibility that should be upheld in XML schema design. A successor effort, METS, was launched in 2001 to try to improve on the MOA2 DTD. The METS format retained a very similar model for structuring a digital library object (with a few extensions to support time based media), but implemented a far more permissive structure with respect to descriptive and administrative metadata and was created using an XML Schema, rather than a DTD, in order to exploit XML namespaces and their further promise of modularity and schema reuse. The general structure for an object such as a digitized book in METS might look like this:

<?xml version="1.0" encoding="UTF-8"?>
<mets xmlns="http://www.loc.gov/METS/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xlink="http://www.w3.org/1999/xlink"
xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/mets.xsd">
    <dmdSec ID="DM1" >
        <mdWrap MDTYPE="OTHER">
            <xmlData><meta>Descriptive Metadata for Book</meta>
            </xmlData>
        </mdWrap>
    </dmdSec>
    <amdSec>
        <techMD ID="AM1">
            <mdWrap MDTYPE="OTHER">
                <xmlData><meta>Administrative metadata applicable
                to TIFF files</meta></xmlData>
            </mdWrap></techMD>
    </amdSec>
    <fileSec>
        <fileGrp>
            <file ID="F1" ADMID="AM1"><FLocat LOCTYPE="URN"
            xlink:href="urn:x-mets:Location_of_Page_One_TIFF_Image"/></file>
            <file ID="F2" ADMID="AM1"><FLocat LOCTYPE="URN"
            xlink:href="urn:x-mets:Location_of_Page_Two_TIFF_Image"/></file>
            <file ID="F3" ADMID="AM1"><FLocat LOCTYPE="URN"
            xlink:href="urn:x-mets:Location_of_Page_Three_TIFF_Image"/></file>
        </fileGrp>
    </fileSec>
    <structMap>
        <div TYPE="book" DMDID="DM1">
            <div TYPE="chapter">
                <fptr FILEID="F1" />
                <fptr FILEID="F2" />
                <fptr FILEID="F3" />
            </div>
        </div>
    </structMap>
</mets>
        

The goal of representing the structure of a work as a hierarchy of nested <div> elements with TYPE attributes was to have a relatively simple, abstract hierarchical structure that could be readily applied to a variety of materials. This was intended to promote the adoption of the standard (a single, simple standard is more likely to be adopted than a variety of complex ones), which in turn was seen as promoting interoperability. Having all of the digital library community using a single standard for structuring content and metadata would certainly be preferable to the community adopting a variety of disparate standards.

It should be noted that this move towards abstraction was a relatively significant break from the SGML design practices that many research libraries had been using to date. While it is true that the notion of using nested <div> elements was derived from the TEI text apparatus, TEI does not rely on pure abstraction; one does not expect to encounter encoding such as <div type="figure"> in a TEI document, when a <figure> element is available to use. Just as XML itself, the METS schema carries an inscription of its designers' worldview, that it was preferable to develop a single, simple, generalizable, highly abstract model and encoding mechanisms to structure content and metadata for digital library objects, rather than to pursue the development of a variety of highly specific schemas (one for photographs, one for journals, etc.), or a grand encompassing schema that contained elements appropriate to different genres that could be combined as needed (e.g., the TEI model). Despite its use of the abstract <div> element with a TYPE attribute to represent the structural components of a digital library object, however, the METS schema insisted on the use of more specific concrete elements to identify different forms of metadata, with the <dmdSec> element used for descriptive metadata and the <amdSec> element used for administrative metadata, along with a series of subelements for different forms of administrative metadata (technical, rights, provenance and source). This typification of different forms of metadata was itself an effort to promote modularity in further metadata schema development. By identifying specific subclasses of metadata within the METS schema, METS' designers hoped to encourage XML developers in the digital library world to create discrete, specialized metadata standards that would align with those subclasses, and that those encoding digital library objects could then select from a set of such modular XML metadata standards in composing a particular object. Through METS' design, its implementers consciously sought to encourage the adoption of modular schema design practices within the digital library community.

The digital library community was obviously not the only community in the late 1990s that recognized the potential benefits of XML, nor was it the only community to develop standards for structural metadata based on this notion of a highly abstract structure. In October of 2000, the Moving Picture Experts Group (MPEG) issued a call for proposals for "digital item declaration." This work culminated in the release of the MPEG-21 Digital Item Declaration (ISO/IEC 21000-2) standard in 2003. While MPEG-21 is similar to the METS standard in employing a rather abstract hierarchical structural mechanism for ordering content and metadata, it differs inasmuch as non-structural metadata (<Descriptor> elements in MPEG-21 parlance) are not typed, and structural metadata elements, while still rather abstract, are of three different types: <Container>, <Item> and <Component>. Our METS example above, if translated into MPEG-21 DIDL format, might take the following format:

<DIDL xmlns="urn:mpeg:mpeg21:2002:01-DIDL-NS"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:mpeg:mpeg21:2002:01-DIDL-NS
file:/Users/jeromemcdonough/Documents/GSLIS/Research/OAIORE/MPEG21.DIDL.OFFICIAL.xsd">
    <Container>
        <Item>
            <Descriptor id="DM1">
                <Statement mimeType="text/xml"><meta>Descriptive
                Metadata for Book</meta></Statement>
            </Descriptor>
            <Descriptor id="AM1">
                <Statement mimeType="text/xml"><meta>Administrative
                Metadata Applicable to TIFF Files</meta></Statement>
            </Descriptor>
            <Item>
                <Component>
                    <Descriptor><Reference target="#AM1" /></Descriptor>
                    <Resource mimeType="image/tiff" 
                    ref="urn:x-mets:Location_of_Page_One_TIFF_Image">
                    </Resource>
                </Component>
                <Component>
                    <Descriptor><Reference target="#AM1" /></Descriptor>
                    <Resource mimeType="image/tiff"
                    ref="urn:x-mets:Location_of_Page_Two_TIFF_Image" />
                </Component>
                <Component>
                    <Descriptor><Reference target="#AM1" /></Descriptor>
                    <Resource mimeType="image/tiff"
                    ref="urn:x-mets:Location_of_Page_Three_TIFF_Image" />
                </Component>
            </Item>
        </Item>
    </Container>
</DIDL>
        

While it obviously uses somewhat different element names and linking mechanisms than METS, MPEG-21 DIDL follows a very similar model, and has been successfully employed by several digital libraries. The logical structure of the object is laid out via a hierarchical arrangement of <Container>, <Item> and <Component> elements, and various nodes in that tree structure can be associated with metadata via <Descriptor> elements, and content via <Resource> elements.

If we examine the MPEG-21 standard to determine what inscriptions its designers have placed within it regarding its use, we find messages very similar to that of METS. Structural metadata should be highly abstract, so that a very small set of elements can be employed to structure widely disparate content genres. While METS was perhaps more vocal in trying to push the message that further development of metadata schemas should try to create small, focused and modular metadata sets that could be drawn upon as needed to encode a particular object, MPEG-21's use of <Descriptor> elements that can be associated with other elements throughout an MPEG-21 document conveys the same message. Other structural metadata standards of interest to the digital library community employ similar mechanisms. The XFDU standard for data archiving uses hierarchies of <ContentUnit> elements that may be associated with <dataObjects> and <metadataObjects>. The IMS Content Packaging standard for learning objects uses hierarchies of <item> elements that may be associated with <resources> and <metadata>. While implementations differ in details, the pattern is similar and widespread across the various structural metadata standards of concern to the digital library community. Again and again we designers seeking to achieve wide adoption of their standard in order to promote interoperability between differing institutions; to secure this goal, they favor a highly abstract structural mechanism which can be applied to a wide variety of content, and mechanisms to allow a variety of additional metadata schema to 'plug and play' within the larger structural framework.

While of perhaps some limited interest to social researchers of technology, none of the preceding seems particularly surprising or problematic. That the designers of XML itself, and of encoding standards for digital library metadata and content, should favor flexibility, extensibility, modularity and the use of abstraction to support the generalizability of their standard, and hence promote its widespread adoption to help achieve interoperability, would not be a great shock to anyone who has spent more than five minutes in the company of technologists. These are all considered almost innate goods among software engineers in general and markup language enthusiasts in particular. Yet the NDIIPP tests cited previously would seem to indicate that flexibility, extensibility, modularity and abstraction are not in and of themselves sufficient to achieve interoperability. So what, specifically, is the problem that METS and other structural metadata standards are encountering?

Why is XML like a rope?

One of the earliest discussion points in the development of the METS standard was which of the various elements within the schema should be declared mandatory and which optional. After some discussion among the members of the working group that established METS' original design, it was decided that the <structMap> element, which records the basic tree structure on to which content files and metadata are mapped in METS, would be the only required element. METS, in the group's opinion, was fundamentally a structural metadata standard; it existed to provide a framework into which other descriptive and administrative metadata, as well as content, could be placed. The <structMap> element provided the tree upon which all of the other structural components of METS where hung, where the logical and physical structure of a work could be delineated, and so was really the core of a METS file and the only section that needed to be mandatory. As the <structMap> was the only mandatory portion of a METS file, it was also expected that any structural description of a work should reside there; software that would process a METS file would expect to find logical or physical descriptions of the structure of a work residing within a structural map, and not elsewhere in the METS file.

It was a matter of some surprise for many in the METS community, then, when the Library of Congress, which serves as the maintenance agency for the METS standard, began to produce METS files for digital versions of certain kinds of audio recordings which placed the logical structure of the works in MODS records contained within the METS descriptive metadata section (<dmdSec>) rather than in the structural map, and registered a profile of METS establishing this as their formal internal practice for 'recorded events' (Library of Congress, 2006). The MODS record within a METS file would provide a logical structure for the work using a hierarchical arrangement of the MODS <relatedItem> element, while the METS <structMap> would contain the physical structure, with ID/IDREF links used to draw connections between the two structural descriptions. A recorded concert, for example, might have a MODS record containing a hierarchy such as this^[5]:

<mods:mods xmlns:mods="http://www.loc.gov/mods/v3"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.loc.gov/mods/v3
http://www.loc.gov/standards/mods/mods.xsd">
    <mods:titleInfo>
        <mods:nonSort>The </mods:nonSort>
        <mods:title>1946 Library of Congress recital</mods:title>
    </mods:titleInfo>
    <mods:relatedItem type="constituent" ID="DMD_disc01_tr001">
        <mods:titleInfo type="uniform"><mods:partName>Chaconne
        von Vitali</mods:partName></mods:titleInfo>
    </mods:relatedItem>
    <mods:relatedItem type="constituent" ID="DMD_disc01_tr002_005">
        <mods:titleInfo><mods:title>Sonata in G minor, BWV 1
        001</mods:title></mods:titleInfo>
        <mods:relatedItem type="constituent" ID="DMD_disc01_tr002">
            <mods:titleInfo><mods:partName>Adagio</mods:partName>
            </mods:titleInfo>
        </mods:relatedItem>
        <mods:relatedItem type="constituent" ID="DMD_disc01_tr003">
            <mods:titleInfo><mods:partName>Fuga : allegro</mods:partName>
            </mods:titleInfo>
        </mods:relatedItem>
        <mods:relatedItem type="constituent" ID="DMD_disc01_tr004">
            <mods:titleInfo><mods:partName>Siciliano</mods:partName>
            </mods:titleInfo>
        </mods:relatedItem>
        <mods:relatedItem type="constituent" ID="DMD_disc01_tr005">
            <mods:titleInfo><mods:partName>Presto</mods:partName>
            </mods:titleInfo>
        </mods:relatedItem>
    </mods:relatedItem>
    <mods:relatedItem type="constituent" ID="DMD_disc01_tr006">
        <mods:titleInfo><mods:title>Paganiniana : variations</mods:title>
        </mods:titleInfo>
    </mods:relatedItem>
    <mods:identifier type="lccn">99594334</mods:identifier>
</mods:mods>
            

while the structural map would record the physical structure of the work as follows:

<div TYPE="cd:compactDiscObject" DMDID="MODS1">
    <div TYPE="cd:disc">
        <div DMDID="DMD_disc01_tr001" TYPE="cd:track"></div>
        <div DMDID="DMD_disc01_tr002" TYPE="cd:track"></div>
        <div DMDID="DMD_disc01_tr003" TYPE="cd:track"></div>
        <div DMDID="DMD_disc01_tr004" TYPE="cd:track"></div>
        <div DMDID="DMD_disc01_tr005" TYPE="cd:track"></div>
        <div DMDID="DMD_disc01_tr006" TYPE="cd:track"></div>
    </div>
</div>
        

This represents a valid use of METS (in the technical XML sense), but is a departure from expected practice, which would be to include two structural maps within the METS file, one representing the physical structure and the second representing the logical structure of the work.

This example demonstrates two problems that have been impeding the development of interoperable content within the digital library community. The first is that the implementation of highly abstract elements for the definition of structure provides a tremendous amount of flexibility to document encoders; there is a vast number of potential encodings of any given object in METS, with variations possible in depth of structure (do I limit my structure to musical movements or do I provide structural information to the measure level?), labeling (you say TYPE="book", I say TYPE="monograph"), and arrangement (should the Lord of Rings film trilogy be encoded as a single METS file? Three METS files? Three METS files for the individual films and a fourth representing the abstract notion of the Trilogy?). This can lead to significant variation in encoding practices, even between two institutions dealing with remarkably similar material and using the same metadata standards, as noted by (DiLauro et al., 2005).

The second problem is what we might call the problem of standards independence. The various metadata schema standards which have been developed with the help of the digital library community during the past decade have all been created with the understanding that in order to ensure their usefulness in a variety of different organizations and environments, they should not contain inherent dependencies on other schemas; they need to be able to express all of the information necessary for their particular domain on their own. For many of these standards, the designers recognized a need to be able to account for relationships between various content objects being described by the standard, whether the description being applied was the more traditional form of intellectual description you would expect in a library catalog, or a technical description of the composition of a TIFF image. The result has been that a number of common metadata schemas within the digital library field contain elements for expressing structural metadata: Dublin Core has its <relation> element, MODS has its <RelatedItem> element, the PREMIS schema for preservation metadata has a <relation> element, even the MIX standard for still image technical metadata contains an element for referencing previous image metadata. As the standards' developers felt they should not make their efforts dependent on structural metadata mechanisms in other standards, they implemented their own. Unfortunately, with the addition of each new metadata standard containing structural metadata capabilities, the potential for difficulties with our first problem increases. Every new metadata standard within the digital library community seems to add another mechanism for describing the structural relationships between content objects, and hence greater potential for variation in object encoding practices.

The irony is that both these problems derive from the flexibility, extensibility, modularity and use of abstraction to create structural metadata elements of wide applicability that the designers of the metadata schemas hoped to promote. The potential range of variation in encoding structural metadata is the result of each of these factors. The use of abstraction in METS (i.e., the <div> element) was an attempt to make the standard flexible in application; however, it opens up a tremendous degree of play in encoding practice. If you ask two different individuals how many page breaks there are in a text, the likelihood that they will get the same answer is a good deal greater than if you ask them how many divisions there are in a text. The use of abstraction opens up encoding to a much greater degree of personal interpretation, and hence variation. The extensibility of METS and the hope to promote a modular system of metadata schema reuse that its authors inscribed within it open up the possibility of using other metadata schemas to encode structural metadata. And it was this same desire for flexibility and modularity that has led other metadata schema designers to include structural metadata components in their own schema; they wanted to ensure that their own efforts were flexible enough to be applied in a variety of settings, and with a variety of other schemas. But having to design their own schema without knowing the specific supporting capabilities to be found in other schemas with which their own might be used, they are inevitably forced to create structural metadata capacities of their own within their schema. The designers of metadata schemas (structural or otherwise) within the digital library community have sought to adhere to a particular set of design practices, seeking to create flexible, extensible, modular and generalized tools, and to promote like practice in others through inscription of their view of appropriate XML design within their technological artifacts. Unfortunately, promoting such good practices has been a death blow to one of the principle reasons for adopting XML in the first place: to ensure interoperability of digital library materials across systems. wide-scale interoperability requires wide-scale adoption, but the design practices of schema implementers intended to promote wide-scale adoption run directly counter to wide spread interoperability.

Hence XML's similarity to a rope. Like a rope, it is extraordinarily flexible; unfortunately, just as with rope, that flexibility makes it all too easy to hang yourself.

Where do we go from here?

The digital library community seems to face a dilemma at this point. Through its pursuit of design goals of flexibility, extensibility, modularity and abstraction, and its promulgation of those goals as common practice through its implementation of XML metadata standards, it has managed to substantially impede progress towards another commonly held goal, interoperability of digital library content across a range of systems. How then, should the community respond?

One possible response to this situation would be to say that perhaps our community cares less about interoperability than we thought. Despite the existence of projects such as the DLF Aquifer venture, it may be that interoperability is in fact a lower priority for the digital library community than it likes to believe, and the adoption of metadata standards that impede interoperability is merely a reflection of that underlying reality, and not a major problem to resolve. There is at least some reason to suspect this may be the case. Most research libraries have a clearly defined local clientele, and while voices within the digital library community have been calling for the liberation of content from local silos for some time (Seaman, 2003), libraries' primary responsibility will always be to their local communities. If systems developed to deliver digital library content to the local community are successful in that context, and if the costs associated with achieving much more widespread interoperability are high, then many libraries may decide that interoperability, while desirable, is a goal which may have to wait.

If libraries do wish to make progress on the issue of interoperability of structural metadata, they will need to recognize that, as (Renear and Golovchinksky, 2001) observed, "every significant information processing standardization effort must skillfully negotiate competing and apparently irreconcilable objectives, [and] serve a wide variety of stakeholders with many different interests." In the case of structural metadata, the particular competing objectives that the digital library community does not seem to have successfully reconciled to date are what (Kendall, 2007), in a discussion of blogging practices, has labeled the problem of "control vs. connection." The structural metadata standards which have been developed to date, with their emphasis on flexibility, extensibility and modularity have sought to afford local institutions the greatest degree of control possible in their encoding practices. The standards are designed to allow any given institution to do what it wants. This has obvious benefits in terms of easing adoption of the standard in any given context, and as a result insuring the standards' widespread adoption (obviously a good thing in a standard). However, increasing the amount of local control over the ways in which a language is used and developed is fundamentally at odds with a language's ability to serve as a means for connection with others outside the local context. It is, in essence, promoting the development of regional dialects at the expense of mutual intelligibility. The particular case of structural metadata standards reveals that sufficient local variation in syntax, the ways in which people structure their objects using a markup language, can be as fatal to communication as variation in semantics.

Given this fundamental tradeoff between control and connection, libraries wishing to promote interoperability of digital library content have two possible strategies. The first, and most obvious, is to attempt to alter the balance currently struck between connection and control to more strongly favor connection. There are several mechanisms which the library community might employ in pursuit of this strategy, including the design and use of schema which more significantly restrict both the means for recording the structure of objects and the ability to employ arbitrary additional schema within instance documents (or developing profiles of existing schema to achieve the same ends), establishing formal rules of structural description (equivalent to rules of description used in cataloging for creating bibliographic records) dictating aspects of object encoding not susceptible to enforcement through XML's validation mechanisms, and mandating the use of particular controlled vocabularies and ontologies within document instances to record information such as a <div> element's TYPE attribute in METS. Decreasing the possibility for local variation in encoding of structural metadata will certainly help improve digital libraries' capability to interoperate with each other. However, removing local capacity for variation will also tend to reduce the number of institutions who are willing to use such a markup language. If the digital library community, for instance, was to revise the METS standard to forbid any use of a <relation> or <RelatedItem> element in a descriptive metadata section to express the logical structure of a work, it would assist in insuring interoperability of digital content, but it might also very well mean losing the Library of Congress's support for the standard. More importantly, however, such an approach seems to overlook one of the fundamental realities of the web environment: communities of practice no longer operate in splendid isolation from each other (if indeed, they ever did). Even if libraries could agree on a structural metadata standard that enabled a significantly greater degree of support for interoperability than we find with today's standards, libraries must now interact with a variety of other communities (publishers, museums, archives, educational technology companies, etc.) that are also creating their own structural metadata standards. The odds of our moving past this structural tower of Babel to achieve a single object encoding practice across all communities do not look promising, to say the least.

The second, and I believe more promising, strategy is to accept that the need for local community control over encoding practices is a valid one, that regional 'dialects' of markup languages are inevitable, and that we must find ways to facilitate information exchange across the boundaries of different communities' markup vernacular. However, this will require a significant shift in the digital library community's relationship to the notion of standards. Specifically, I believe the library community needs to shift from its current singular focus on schema development to a dual focus on both schema development and translation between schemas. It is worth noting here that the Library of Congress currently serves as maintenance agency for a variety of XML standards developed within the library community; if you examine the list of standards that they are maintaining (Library of Congress, 2008), however, you will find that while there are several metadata standards listed, standardized stylesheets to enable conversion between formats are not listed here.

Such stylesheets do exist in some cases. The Library of Congress has, for example, provided stylesheets to enable conversion of MODS descriptive metadata records into MARC/XML format and back. These efforts to formalize prior work that established cross-walks between different descriptive metadata standards are not, however, seen by the community as having the status and importance of standards, as exhibited by their omission from the 'Standards at the Library of Congress' web page. If the digital library community wishes to support interoperability while simultaneously affording institutions localized control over encoding practices, that situation needs to change. Enabling translation activity must be seen as important, if not more important, a standards activity than the creation of schema for metadata sets.

A heightened emphasis on standardizing translation between markup languages will obviously mean further work on formalizing translations between markup languages using XSLT, and treating those with the level of attention and care that the community has lavished on metadata schema. However, it might also be worth considering whether the notion of formal rules of structural description mentioned earlier might be of benefit in trying to achieve greater translatability between different markup languages. As an example of what this might mean, consider the example of the 1:1 principle in Dublin Core (Hillmann, 2005), that a single Dublin Core record should describe one and only one resource. The 1:1 principle provides guidance on the relationship between a metadata record and a described resource that is applicable outside the realm of Dublin Core; in fact, several other descriptive metadata standards developed since Dublin Core have made reference to the 1:1 principle as a guide to usage. We could easily envisage similar principles being developed for structural metadata that could guide usage of a variety of different structural metadata standards, and by working to insure similar use practices, would help insure ease of translation between different structural markup languages. We might, for instance, take as a working principle that any given structural metadata document should never contain more than two levels of structural hierarchy. Our METS example above passes muster with this rule; if, however, we modified it so that a third level of <div> elements was needed (of TYPE 'subchapter', for example), then we would be in violation of this principle. To fix this problem, we could employ METS' <mptr> element to allow the <div> elements for each chapter to reference separate METS files containing the structural descriptions for the individual chapters. Through the establishment of common principles of structural encoding and standardized stylesheets for translation, we might be able to improve our ability to interoperate while simultaneously retaining some flexibility for local encoding practice (although obviously adoption of common principles of structural encoding may impede local control in favor of connection to some degree).

The rise of the network information society is presenting libraries with a variety of new challenges. Perhaps the most significant of these is the heightened degree of interaction with communities of practice that do not share libraries' standards, practices or values. If libraries are to survive and thrive in this new information society, they must alter their own value structure to prioritize communication with other communities to an equal, if not greater, extent than internal communication between libraries. If they pursue this course, they may find that issues of internal interoperability of library systems are more tractable than they have appeared to date.