Balisage Paper: Tracking Toys (and Documents)

Identifying the Box

Like the cardboard boxes with toys, documents need to be named so their contents can be found later. Usually, the name tells us what the contents are about, so reasonable is to include the name of the product being described. If there are variants or models (car models, say), those might need to be included, too.

And, just as with the cardboard boxes, a type identifier might prove useful. Cars, planes, formula one, user guide, and spare parts are all reasonable labels.

But is the maker or manufacturer important to include in a name? Quite a few documents out there probably say things like my file, but is that a good idea?

Surely, a document should have a description of what it is about in its name. A manual for assembling a toy car, for example, should probably have that in the name: Jaguar E-Type Model, Assembly Instructions. But what if the manual is produced in a modern fashion, in modularised form, where the toy manufacturer writes everything in reusable modules and so, the section about the necessary tools would be common to not only model cars but probably model ships, planes and tanks.

Identifying Change

What if the assembly manual was regularly updated to include the changes to the model car? How would those changes be identified? Should they be in the name?

Identifying change is about helping readers to know which version of a document to get. Would it therefore make sense to include a version label in the name? Would it be enough to use ordinal numbers like Manual 1, Manual 2, and so on? Or would it be better to date the new versions and include that?

Personally, I get nervous if I see a version label on a cover. Do I have the right version? Do I have the latest? What version is the product? I always end up thinking wouldn't it make more sense to make sure that the reader got the right version to begin with? Versions are needed by the writers to know which version to attach to which product, and which version to translate; the readers have no real use for the information.

But on the other hand, consider that box of old cars mum put in the attic: if it only said OLD TOYS on the lid and there were others like it, some considerable work would be required if you ever wanted to get rid of some of your old stuff. Rummaging through the attic is never any fun.

So the answer is both yes and no. It's useful to know which one of the two versions of a document you have in front of you is the old one, but at the same time, there is no way for the reader to know if there is a still later version than the two that you have.

Identifying Context

If the model car assembly manual was translated to a number of languages, should the target languages be included in the name? Is it relevant to include, say, Swedish, in the name?

The language identifies a context in which the manual is intended to be used; often, a language and country combination is needed to identify not only the language but also the intended market. Swedish, for example, is a language spoken in both Sweden and Finland, and while the language is mostly the same in both cases, the countries are not.

Or consider Märklin model train sets. They come in different sizes, so getting the right size for your model railway is of vital importance.

Unlike the version labels, I would argue that identifying the context is important information to the user and should be included.

Contexts as Renditions

Consider an image of that E-Type Jaguar model. If printed, the image's visual contents and its name are one and the same if you know your cars, but a short, descriptive caption (Jaguar E-Type) would still be needed to identify it for someone who doesn't.

Similarly, a version (1962 or 1970, say) will be useful for anyone without the necessary knowledge.

If you are a kid and want it up on the wall, however, the context needs to be paper and that's about it. But what if you need to print it from your computer? Obviously, if you are a kid, you won't care much about the file format—the contents are important, not how they are rendered—but to get a sufficiently high resolution using the right software, in other words, when you need to achieve something specific with the contents, the rendition becomes crucial.

All this is equally true with documents. The name describes the abstract contents of the container, the version how they change over time, and the language (and, frequently, country, region or other parameter affecting the exact context) how they are rendered.

The Semantic Document

Or, expressing the above in a more concise manner:

BASE:VERSION:RENDITION

There is no right or wrong here. It could, for example, be argued that Mondeo is a Ford version rather than part of the base name.

IDs

The observant reader will no doubt have thought about identifiers beyond those naming the container by now. What about identifying specific parts of the contents? In toy parlance, the box CARS might contain smaller boxes, say, one for each car, and so there would be a Ford Mondeo MY2008 EU Market, left-hand drive box, a Jaguar E-Type MY 1961 Roadster box, and so on.

I'd argue that the definition of a container is mostly an arbitrary one and will likely change over time. The contents are important and should be considered, but above all, we should always be able to uniquely identify the container, whenever and however it changes. What's inside is uniquely identifiable by first identifying the container and then the particular object of interest inside (the Jaguar E-Type stored in CARS).

Jaguar E-Type is what we call a structural identifier; if the boy's got several E-Types but only one in CARS, then the structural identifier doesn't need to be unique across the toy collection, only inside CARS; the context is enough. Also, with only one E-Type inside CARS, we don't actually have to use the full identifier to find it even if it does have one.

On the other hand, if that E-Type proves to be the boy's favourite toy, he might well want to store it in its own box, separately from the others. Once outside, it would have to be referred to using its full identifier, especially if there were other Jaguars. The boy, of course, would probably just refer to it as the Jag^[4].

And if the boy's list of favourites grows over time, he might want to create a favourite cars box and put it there, in which case a structural identifier combined with the unique container name would again be enough.

Reasons for Change

A new version signifies a change and should happen whenever there is a significant change to the contents (begging the question what is a significant change?). When the E-Type toy is moved to its own box, or perhaps to that box with favourites, the old box changes and gets a new version^[5].

The definition of change is a bit like the definition of done to agile practitioners or starving artists, and so hard to offer here. I tend to go with anything beyond pretty-printing a document.

My definition of change, of course, means that there will be a lot of versions, most of which are either significant only to me or not significant at all, beyond me hitting Save. A precious few are of interest to others, and this is where workflows come into play.

A change in the workflow is a status identifier: ready to be reviewed, approved, rejected, ready to be translated, published, etc. A document could move from draft to published without change and thus in a single version, but on the other hand, reaching published might just as well require two dozen versions. None of the versions between the two workflow stages is of any interest to anyone else than those involved. Workflows, then, can be used to scope a name (only use published versions).

Links

Finding the E-Type Jag is achieved using a link. The boy will not think of it as such, but when warning his friend to not touch the Jag in the CARS box, a link is what he uses to get his message across.

A link, of course, would build on the same rules for applicability and scoping as the name itself. However, a link used in an editing situation (don't touch the Jag on the floor) would depend on context while a more stringent link to an outside observer (mum, don't touch the E-Type Jaguar in the CARS box if you go into my room) is meant to be persistent and needs to be better defined.

Versions

Versions indicate change, so in the car world, this would actually correspond to manufacturing weeks in addition to model years (as opposed to actual years). For example, MY2003 expressed using this schema might be M2003 w46 (in reality a model that started production in 2002), MY2003 w12, MY2003 w24, and MY2003 w38. MY2004 would happen in 2003, week 46.

A versioning schema for the Volvo V70 might then be defined as 2003:46, 2003:12, 2003:24, etc. Of course, what the end user sees is still 2003.

What this means is that the version label seen by the customer need not be the same as the version seen by the service technician. Versions, then, are better off scoped. Workflow stages (see section “Reasons for Change”) can help, but so can multilevel versioning (see section “Multi-level Versioning and Business Rules”) and naming abstractions (see id-naming-abstractions).

Translations

Frequently, problems with translations really are about the lack of proper versioning and/or naming. Consider, for example, multinational companies producing content for multiple markets and multiple languages. A manual may be first written and published in one language, say, Swedish, but then translated to another language and market, say, China and APAC, where the contents are further developed before publication to accommodate product changes specific to that market.

After some time, the product is then sold to a third market. It is translated from the second, and customised once again to meet the requirements of that market.

Meanwhile, the original Swedish version has been updated, independently from the other markets, and if they, at that point, decide to use product features developed for the other markets, the documentation from the other market(s) will not be reusable.

This is where the naming schema and regarding translations as renditions will help. Returning to the car model year and manufacturing week example, translations would be based on a specific exact version (rather than a scoped one), so 2004:46:sv-SE should be equivalent to 2004:46:en-GB rather than simply MY2004 sv-SE to MY2004 en-GB. The exact version schema would ensure equivalence^[7]. These would be different renditions of the same basic abstract manual.

Note that the rendition is simply defined as such; a GB manual would describe a right-hand drive car and therefore not be an exact match. Similarly, the GB model might have any number of standard features not present in the SE version or vice versa (a Spanish car would, for example, probably not have heated seats as standard).

DocA en-GB v1 written =>    DocA fr-FR translation of v1
                            DocA fr-FR v1-1
                            DocA fr-FR v1-2
                            DocA fr-FR v1-3
                            ...

DocA en-GB v2
...

DocA en-GB v1 written =>    DocA fr-FR translation of v1
                            DocB fr-FR v1
                            DocB fr-FR v2
                            DocB fr-FR v3
                            ...

DocA en-GB v2               DocB fr-FR v9

Modularisation

Let's say that DocA:2:en-GB (see above) is modularised into a root file and two modules, like so:

DocA:3:en-GB (root; modularisation is a change so a new version)
|
|--DocC:1:en-GB
|--DocD:1:en-GB

When the new DocA is translated to French, this happens:

DocA:3:fr-FR
|
|--DocC:1:fr-FR
|--DocD:1:fr-FR

In other words, the root module and both of the new modules are translated. The root, of course, might simply comprise links to the modules and so the translation might happen automatically.

If a new document, DocE, is written and reuses modules like this:

DocE:1:en-GB
|
|--DocC:1:en-GB
|--DocF:1:en-GB

The translation to french would be

DocE:1:fr-FR
|
|--DocC:1:fr-FR
|--DocF:1:fr-FR

The root, if comprising links only, would be automatically translatable, simply by changing the links from the en_GB renditions to the fr-FR renditions; DocC:1:fr-FR would already exist, as it was translated in the previous run; and so the only thing remaining would be to translate the new DocF:1:en-GB module.

Too Many Versions?

Mentioned above: a new version happens when there is a significant change in contents. Would this mean that a document that was properly modularised from the start would be hell to keep updated and translated? After all, if DocC:1:en-GB is updated to DocC:2:en-GB, doesn't that mean that DocE needs to be updated so the link points to the new version (with the same applicable to every other document using DocC)?

So if this happens:

DocE:1:en-GB
|
|--DocC:1:en-GB
|--DocF:1:en-GB

is reworked to

DocE:192:en-GB
|
|--DocC:28:en-GB
|--DocF:45:en-GB

How many times does the root document need to be updated? Translated? 2? 5? 40? 100?

There are several problems in play here:

The real problem here is the assumption that every version is important. Most aren't, as mentioned in section “Versions”. Let's have a closer look.

Multi-level Versioning and Business Rules

Remember, a translation is a predefined rendition. I.e. we are saying this content is equivalent to that content, not that this content is identical to that content. So use several levels of versioning to scope a link. Assuming these versions:

1.0.0
1.0.1
1.0.2
1.1.0
1.1.1
1.2.0
1.2.1
1.2.2
2.0.0

Define business rules saying that a new translation, or link, is only required for integer versions (or decimal versions, or ...).

This, of course, is equally applicable to modularisation without translation. We could simply say that a link made to DocC v 1.0.2 would be valid for all 1.x.x versions and the business logic could then automatically use the latest 1.x.x version. The same would be applicable for translations.

EU Example - Minimising Work

The multilevel versioning provides an easy solution to the EU translation example, while retaining the all-important link between renditions. A business rule could simply say that every 1.x rendition is equivalent to every other 1.x rendition.

A clever modularisation of the document could then minimise the impact in terms of the number of member country-dependent changes by placing the contents requiring updates into a single module.

Let's say we have this slightly more complex document:

DocG:3:en-GB
|
|--DocC:1:en-GB
|   |--DocX:1:en-GB 
|
|--DocD:1:en-GB
|
|--DocH:5:en-GB
|
|--DocM:7:en-GB
|   |--DocX:1:en-GB 
|
|--DocP:3:en-GB
    |--DocX:1:en-GB 

Here we have DocC, DocM and DocP all linking to the common module, DocX. Of course, the links would not reuse the entire document, only selected parts, expressed as fragment identifiers like so:

DocC:1:en-GB => DocX:1:en-GB#id1
DocM:7:en-GB => DocX:1:en-GB#id2
DocP:3:en-GB => DocX:1:en-GB#id3

This results in limiting the member country-dependent changes to a single module, DocX, that could be updated when required. The links from the source documents would all point at the integer version 1 of DocX, meaning that every update within the 1.x series could be defined as an equivalent link in the business rules.

Another Aside on IDs

Document IDs need to be unique, of course. How else would we find the right box? Structural IDs, however, do not. Internal links (#id) would be unique inside the module, but there is no need to enforce uniqueness outside them since any pointers outside include the document ID that is unique:

DocC:1:en-GB => DocX:1:en-GB#id1
DocM:7:en-GB => DocX:1:en-GB#id2
DocP:3:en-GB => DocX:1:en-GB#id3

Problems may ensue when merging modules or when modularising documents; in both cases, there is a need to check for uniqueness and possibly handle broken links from elsewhere.

Weaknesses

In spite of the precise versioning capabilities, the system had several main weaknesses when it was built:

RootX-v2 => ModuleA-v5 => ModuleB-v9 => Image-v1

RootY-v1 => ModuleA-v8 => ModuleB-v9 => Image-v1

Improvements

The system, then, actually had issues with several concepts introduced above. Does this mean that the idea of a semantic document, labelled using a BASE:VERSION:RENDITION schema, doesn't work after all?

Profiling

Efficient modularisation of documents require being able to profile the contents, marking up sections to be about the different variants covered by the module and then showing and hiding them when publishing the document, depending on the publishing context. This practice also includes features like variable text strings so a variant name can be included in the content when using a specific profile.

Profiling is beyond the scope of this paper, but I presented a fuller treatment of how to manage profiles a number of years ago. See id-naming-abstractions.

Passive Version Tracking

The ideas described in this paper are all about labelling and annotating container, a play with the labels that identify those toy boxes with cars (or the boxes inside the larger boxes). The cardboard box metaphor is best visualised by the labels taped on the boxes but since it's just a metaphor, there's nothing to stop us from moving the labels to a more convenient location. In other words, indirection, a sort of an out-of-line versioning.

We can place imaginary labels on the boxes, on the cars, on groups of cars, and so on (think registers), so we can also remotely track the changes. If you think about the ideas presented here, they all observe and react on the changes to the content; they do not actually cause them.

FRBR

Two reviewers suggested I should not finish this paper without considering Functional Requirements for Bibliographic Records (FRBR) (see id-frbr). I must confess that FRBR is mostly new to me. I have heard of it before, somewhere, but I never made the connection.

There are similarities between what I propose and FRBR, in that there is a concept that reminds me of the abstract, rendition-less content that is my base document, and there is a concept similar to a rendition (called a manifestation, a phrase I like). There is also what is to some extent an equivalent, or at least similar, to a version (called an expression, I phrase I don't like), but while I think I understand where the authors of FRBR are coming from, I don't think FRBR is an equivalent, and here is why:

FRBR also describes what they call derivative works. These may include, say, annotated versions of a movie script but also any other version of the base work, and while technically, there is nothing in theory to stop an author from including a derivative work such as an annotated script as a new version in what I propose here, my proposal is not about including any kind of derivative work as a version. My versions are very strictly about change to the document, not interpretations of it.

My proposal (and practice; I have implemented what I describe here in several live systems) is about a base document, an abstract document without a context requiring a specific rendition, nor a version identifying its change over time, that then is changed over time, performed in increments where each increment is defined by a user as a significant (and therefore saved) new version, and where any of those user-identified versions can then be rendered in a specific way, depending on context.

It's a bit like quantum physics for versioning; I am talking about incremental changes where each change can be rendered according to (again) some user-defined context, with each rendition being defined as equivalent to another, which means that there can be no middle ground between two versions. In FRBR, however, at least in how I interpret it, there is a fluidity between manifestations and expressions, leaving the standard open to connecting not just versions and renditions of the document itself but also any derivative work it might have.

In my quantum versioning, that annotated version could be defined as a fork, which would be perfectly fine, but I would probably not want to mix my semantics in that manner as an annotated work would be a single version (in some rendition or renditions) where some selected nodes had been linked to external content.

In other words, while I can certainly appreciate the similarities between my work and FRBR, I do think the approaches are different.

Why This Paper?

I presented the passive version tracking idea at XML Prague 2016 (see id-xmlprague), expecting questions and objections on the idea itself. It didn't happen. Instead, people doubted the translations-as-renditions definition, both immediately after the talk and over beers. The EU translations example was brought up as proof against the idea and so triggered this paper.

So?

The basic principles outlined in this paper are already in use. The passive tracking system is a proposed proof of concept for a client of mine. Plus, I can't think of a good alternative. Can you?