Balisage Paper: Semantic Profiling Using Indirection

Publishing

Publishing is a simple XSLT transformation that filters the contents according to context. If several conditions with differing semantics are used (such as profile and audience, above), using one stylesheet per profile type is easy to set up in an XProc pipeline used to preprocess the document.

Essentially, filtering is about string matching. The publishing context value needs to match the node's profile for the node to be included^[4].

Boolean Logic

The above examples all assume OR. A single profile value match is enough to include a node. Similarly, even with a multiple-value context, a single matching profile value is enough:

<doc profile="A B">
   <p>Information common to products A, B, and C.</p>
   <p profile="A">Information about product A.</p>
   <p profile="B C">Information about products B and C.</p>
   <p profile="B C">Information about products B and C.</p>
</doc>

Here, nothing is excluded from the output because the A context matches the first node and B the second and the third.

Sometimes, Boolean AND is preferred^[5]. Here is a simple (and limited) way to do it:

<doc profile="B">
   <p>Information common to products A, B, and C.</p>
   <p profile="A">Information about product A.</p>
   <p profile="B C">Information about products B and C.</p>
   <p profile="B C" boolean="AND">Information about products B and C.</p>
</doc>

Here, boolean="AND" in the last paragraph means that for it to be included, every profile value must match the context (B AND C). In this case, the last paragraph is therefore excluded from the output.

More advanced Boolean expressions might be useful, of course, but would almost certainly prove to be far more problematic to implement in an XSLT stylesheet:

<p profile="B AND C AND NOT(A OR D)">Some text.</p>

This is no longer mere XSLT territory; an external tool for parsing the attribute contents might prove necessary.

Editor

To make URNs practical, the writer will need help to identify and insert a profile (while URNs are unique, they are not necessarily human-readable). Similarly, when editing existing profiled nodes, the profiles must be easily identifiable.

The problem, of course, is that a string like urn:x-cassis:r1:cos:xplatform:000359:sv-SE:0.12 is not very descriptive. Identifying it requires asking the CMS, which might prove cumbersome if one ever wanted to work offline.

A cop-out solution is to use strictly human-readable URNs, but problems such as identifying the variations in scope in the D5 example above (see section “Changing Scope”) require more.

Perhaps better and certainly easier to process is to insert descriptive throwaway attributes containing current profile values when checking out or opening a document in the editor. Such an attribute, say, values, would be for convenience only and be stripped from the document at check-in:

<p profile="urn:x-profile:a:sv-SE:12" values="A">Information about product A.</p>

An more powerful alternative requiring a bit more processing is to use a mapping document listing any required profile-and-value pairs for any checked out or open documents, like so:

<maps>
		...
		<pair>
  		<profile>urn:x-profile:a:sv-SE:12</profile>
  		<values>A</values>
		</pair>
		<pair>
  		<profile>urn:x-profile:a:en-GB:12</profile>
  		<values>B</values>
		</pair>
  ...
</maps>

Or some variation thereof. A mapping document might also provide the basis for a profiling GUI, listing the available profiles and their versions in some human-readable form, an immediate advantage being that once populated, the mapping document would give access to the available profiles without requiring a server connection.

I've used a similar approach with a mapping document when matching URNs for checked-out or open documents with their temporary URLs in the editor:

<Repository>
    <RepositoryName>CosTI</RepositoryName>
    <Map>
        <UrnUrlPair>
            <Urn>urn:x-cassis:r1:cos:00002730:sv-SE:0.7</Urn>
            <Url>C:\Users\arin\Documents\condesign\cassis\ti\xmetal\2880321bb5d24b08a95e2854bccf859b\prox-för-cassis.xml</Url>
            <Writable>false</Writable>
            <EditUrl />
        </UrnUrlPair>
    </Map>
    <ShowMetadataDialog>true</ShowMetadataDialog>
</Repository>

Expanding this to include profiling would be relatively easy^[11].

Variable Text and Localisation

Variable text in the editor can be inserted using both techniques above: a throwaway values attribute or a separate mapping document both do the trick. The former alternative requires less processing while the latter gives access to more features. Localised values, for example, would require the mapping document.

Combining Profiles

URNs (and indeed any type of abstraction layer) can help simplify complex profiles, such as the logical expressions mentioned in section “Boolean Logic”. Instead of having to process the expression in an attribute, the expression can be represented using another URN, like so (with apologies for the pseudo-code):

URN-EXPRESSION = URN1 AND URN2 AND NOT(URN3 OR URN4)

The replacement URN represents the expression and is used instead of it when processing. Of course, to be more than a theoretical exercise in neat ways of doing the unneeded, the situations in which boolean expressions can occur must be clearly defined. Such situations are common when describing complex modular products and their many variants; such products are frequently sold as individuals, requiring individualised documentation. A closer look of those situations is outside the scope of this document, but the point I want to make here is nevertheless an important one: rather than processing 2*(3+2), process 10. An abstraction layer is simply some suitable representation of semantics.

Thus, a writer might use a shortcut URN to represent a group of profiles comprising several URNs. Such a user-defined URN could be paired with descriptive metadata to help identify it and other URNs created for similar purposes. The right systems support could easily provide the user with a listing of the underlying profiles.

Base Profiles

A complete profile includes localisation and version information, but sometimes it is useful to process the base profile regardless of language, country or version. This is easily done by defining wildcard behaviour:

URN:*:*

This basically ignores the wildcards; it matches every single one. With the URN semantics well defined (I use EBNF for mine) this should be easy.

Other useful variations here might define processing for, say, the latest version of a profile. A stylesheet treating URN:sv-SE:* as the latest is not hard to do but will, of course, require access to the corresponding values, either at runtime or when populating a mapping document.

Assertions

Sometimes, filtering profiled content causes structural problems in the resulting document, with required elements missing. Consider this admittedly simplistic example:

<doc profile="A">
  ...
  <warning>
    <p profile="A">Some content.</p>
  </warning>
  ...
</doc>

If a warning must always contain at least one p, the above will result in an invalid warning if published in context B rather than A. This is an easy mistake to make, and more complex nodes could easily end up being invalid without the user noticing, especially in modularised documents, resulting in the problem remaining undiscovered until the document is published.

As these problems will only appear later^[12], they can be difficult to spot. This can be solved using schematron (ISO standard; see id-idso-sch) assertions and validation on a document to check for problems and missing content after applying profiles. Such tests can be automated and used to validate the profiled nodes only. Here's a schematron fragment for checking if the warning contents match the publishing context:

<!-- Profiling status for node -->
<pattern>
    <rule context="warning">
        <assert test="p/@profile">No profiling information.</assert>
        <report test="p/@profile">Profiling present.</report>
    </rule>
</pattern>
<!-- Match -->
<pattern>
    <rule context="warning">
        <report test="contains(/*/@profile,p/@profile)">Profile matches 
            publishing context.</report>
    </rule>
</pattern>
<!-- No match -->
<pattern>
    <rule context="warning">
        <assert test="contains(/*/@profile,p/@profile)">Profile does not 
            match publishing context.</assert>
    </rule>
</pattern>

Note that complex schematron documents can be automatically generated if the possible profiles are known and the possible changes are defined in a schema.

It might be possible to use XML Schema 1.1 assertions but since an assertion on an element cannot refer to siblings or ancestors (id-xsdassertions), the assertion would have to be made on descendants only, like so:

<xs:schema 
    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
    elementFormDefault="qualified">
    <xs:element name="doc">
        <xs:complexType>
            <xs:sequence 
                maxOccurs="unbounded">
                <xs:element 
                    name="warning">
                    <xs:complexType>
                        <xs:sequence 
                            maxOccurs="unbounded">
                            <xs:element 
                                name="p">
                                <xs:complexType 
                                    mixed="true">
                                    <xs:attribute 
                                        name="profile"/>
                                </xs:complexType>
                            </xs:element>
                        </xs:sequence>
                        <xs:attribute 
                            name="profile"/>
                    </xs:complexType>
                </xs:element>
            </xs:sequence>
            <xs:attribute 
                name="profile"/>
            <xs:assert 
                test="contains(@profile,.//*/@profile)"/>
        </xs:complexType>
    </xs:element>
</xs:schema>

This might result in some rather complex expressions, if the assertion required needed to go beyond the basics as illustrated above. I have not further explored this at the time of this writing.

Publishing

Publishing documents that include URN profiles remains easy; the URNs can be processed as strings, using string matching, so the filtering of nodes should not be a problem. Processing a translated document that uses untranslated profiles might prove tricky, however. Here is an example of a document originally profiled in Swedish but now translated to English:

<doc profile="urn:x-profile:a:en-GB:12">
   <p>Information common to products A, B, and C.</p>
   <p profile="urn:x-profile:a:sv-SE:12">Information about product A.</p>
   <p profile="urn:x-profile:b:sv-SE:7">Information about product B.</p>
   <p profile="urn:x-profile:c:sv-SE:3">Information about product C.</p>
</doc>

None of the profiled p elements is included in the resulting publication. This, of course, could be the desired result, but more likely is that the profiles need to be preprocessed. One way could be to prep the file going to translation, replacing any language/country information in the URNs before translation. More flexible is to define the exact preprocess according to need. For one thing, if the profiled node is not relevant in the target localisation, the profile should remain unchanged^[13].

Also, the translators should be made aware of any processing requiring exact values (most notably when using profiles for variable text in content); the profile values in a localisation are far more important to the translator than their corresponding URNs. The latter, then, need to be mapped to any relevant values, including values resulting from localisation or from some special processing (i.e. if the latest version of a profile is preferred), before the original document is translated. The values can be placed in a mapping document^[14], provided to the translators but they'd almost certainly prefer preprocessed documents where text variables such as the phrase element in section “Variable Text” include their values rather than the URNs:

<p>Information about product <phrase profile="A B C">A, B and C</>.</p>

The Grammatical Problem Solved

The following sentence using a text variable will potentially cause problems if the number of applicable profiles varies:

<p>The <phrase profile="A B C">is the latest generation diesel engine
for the environmentally conscious driver.</>.</p>

Brute force solutions involving marking up inline content to identify grammatical constructs might be manageable if only two need to be handled, if Boolean constructs are accepted, for example, by using expressions such as profile="(A AND NOT(B)) OR (B AND NOT(A))" for singular and profile="A AND B" for plural form, but even this will quickly become unmanageable for the writer.

Far more useful is to add an abstraction layer that defines the types of profiles, for example, diesel engines or turbo engines. A mapping document might define a group of profiles for the purpose, like so:

<group>
		<profile>urn:x-profile:abc</profile>
		<included>
  		<profile>urn:x-profile:a</profile>
  		<profile>urn:x-profile:b</profile>
  		<values>D5</values>
		</included>
  ...
</group>

Here, all variants are called D5 but the value could just as well be D Series Diesel Engine or something else. The point is that the abstraction is needed to a) group the participating profiles into a meaningful semantic group while b) keeping either singular or plural form, but not both, regardless of the number of exact profiles used.

A different but useful way to solve the problem is to count the context profiles in the root (one or more) and include markup to handle only the grammatically relevant differences. Singular might be marked up as <wrap context="s">is</wrap> and plural as <wrap context="p">are</wrap> or similar.

Basic Requirements for the Document

Some requirements for that third-party document structure emerge:

XLink Roles and Titles Put to Use

The XLink spec describes roles that can be applied to other XLink semantics:

The value of the role or arcrole attribute must be a URI reference as defined in [IETF RFC 2396], except that if the URI scheme used is allowed to have absolute and relative forms, the URI portion must be absolute. The URI reference identifies some resource that describes the intended property.

Sound familiar? A URN, of course, is a type of URI, so roles in XLink can do more or less exactly what I have described above, except, of course, that there can only be one URN per locator. That means that either a set of URN profiles is described in a set of locators, with one URN per locator, or the URNs are combined to other URNs as hinted in section “Combining Profiles”. Of course, that resulting URN is just a reference [that] identifies some resource that describes the intended property, so we seem to be well within the intentions of the spec.

The spec also describes titles:

The title attribute is used to describe the meaning of a link or resource in a human-readable fashion, along the same lines as the role or arcrole attribute.

So, here's an XLink version of the above example (stripping namespace stuff and such):

<linkbase>
  <locator href="URN-DOC:sv-SE:1#id1" role="URN1:sv-SE:1" title="A"/>
  <locator href="URN-DOC:en-GB:1#id1" role="URN1:en-GB:1" title="B"/>
  ...
  <locator href="URN-DOC:sv-SE:2#id1" role="URN1:sv-SE:2" title="A C"/>
  ...
</linkbase>

This, of course, is exactly what we need, and in processable form.

Editor

When profiling a document out-of-line, we are essentially editing a linkbase, that is, a separate document. That document does not need to be edited using full XML editing capabilities, only what's needed for locating the nodes and profiling them. This suggests a DITA map-like approach. In quite a few XML editors out there, there is a separate window or pane for editing DITA maps. It is specialised and only needed to handle editing topicrefs and such.

Similarly, a linkbase editor needs only include the necessary profiling handling, adding locators to the linkbase using a function in the main editor and then adding profiles in the linkbase editor. For editing existing locators, it should be enough to click on them to locate the corresponding nodes in the editor but remain in the specialised window when editing them.

The remaining problem is a practical one: how does one visualise a profile in the main editor window so that the user can easily spot any profiled content. Here, a processing instruction might suffice if the editor has trouble populating the document tree from more than one source.

Schematron Uses

Schematron can be used to validate the resulting profiled content and to generate PIs in the target document to indicate profiles, but also to generate the linkbase itself, if the rules describing what can be profiled and how are formalised. If the target document doesn't contain IDs, a schematron-like reporting function implemented in the specialised editor can be used to identify nodes using XPath expressions.

Variable Text

Variable text is more difficult to implement properly out-of-line, not because of the out-of-line approach itself but because we don't necessarily control the document. The language must be such that the profile value naturally fits into the text flow.

If profiling content we don't control, the problem is mostly beyond our control. If there are placeholders, the problem can be solved, but if not, while we can pinpoint a location using a variety of means (anything from a PI to XPointer comes to mind) variable text when profiling out-of-line should probably not be attempted if we don't control the content.

The Grammatical Problem Revisited

The grammatical solution suggested in section “The Grammatical Problem Solved” is even easier to handle in a linkbase, as the profiles are all in one place, regardless of how many modules they are used in. They can easily be preprocessed by wrapping selected groups in abstractions (grouping profiles under a single label), counting them, and otherwise producing any relevant information about them when publishing a document.

A reviewer of this paper commented: Instead of storing the variable text only and using additional wrap element for the grammatical relevant differences, why not store the whole grammatical phrase as variable text?

This is perhaps an easier solution for an author to handle than the one I started out with, but one that will cause the duplication of any surrounding grammatical phrases and likely reintroduce copy-paste editing when authors include the complete phrases with the variant information, rather than only the product variants themselves. Applying this on the sentence from section “The Grammatical Problem Solved”, we might end up with a number of variants, almost but not quite copies of each other:

And so on, for any permutations that may arise. The conclusion, in my mind, is obvious: any more complex sentences involving named variants in this manner should probably be avoided in favour of a more generic label (say, diesel engine or D, in this case) in any real-world document.