Balisage Paper: On Teaching XQuery to Digital Humanists

Defining the Digital Humanities

So this paper is about teaching XQuery to digital humanists, which raises the question, who are the digital humanists I have in mind? The exercise of defining the digital humanities is among the most popular and, at this point, hackneyed exercises in the digital humanities. I'm sure that I could solicit multiple definitions from the attendees of Balisage. Some of you here helped to bring about the field of the "digital humanities" from the now largely superseded concept of "humanities computing" (see Piez 2008). As far as it goes, I like the definition in the recently-published "Proposal for a Digital Humanities Center at Princeton University."

The definition is instructively broad. On the one hand, it encompasses the whole range of tools that humanists use in the course of their research and teaching. On the other, it indicates that digital humanities involves taking a critical perspective on these tools. In other words, digital humanities is not just about using PowerPoint to display information to a class but also thinking about how PowerPoint as a medium shapes and influences the transmission of that information. Inevitably, digital humanities tends to divide into those who concentrate on the development of computational approaches to humanistic problems and those who prefer to interpolate digital media critically.^[1] The methodologists themselves work across a wide variety of fields and with a large number of tools, including digital text editing, geographical information systems (GIS), natural language processing, network analysis, the semantic web and statistical analysis, to name only a few. Obviously, no single researcher can be competent in all these areas. Accordingly, the digital humanities has been described as a "big tent," though according to Patrik Svensson it might better be described as "a meeting place, innovation hub, and trading zone." Svensson 2012, p. 46 In short, the digital humanities is less a discipline than a meeting place of practitioners of various disciplines with different levels of skill, different modes of production, and differing degrees of critical insight on their work.

The practical point is that one cannot assume anything when teaching "digital humanists." There are no prerequisites to calling oneself a "digital humanist" beyond, I suppose, a general interest in computation and the humanities. As a practical matter, digital humanists hail from many different parts of the academy as well as from outside its cloisters. In a broad sense, administrators, faculty members, graduate students, librarians, alt-ac scholars, and information technologists may all credibly claim to be digital humanists. Their perspectives on digital humanities will differ broadly according to discipline and social-economic location. Professors generally enter the digital humanities with specific research agendas. Librarians, by contrast, do not generally have their own research programs; more typically, they're seeking new ways to provide and improve access to the collections under their care. Graduate students may be exploring the options or working as assistants on faculty projects while contemplating alt-act alternatives to standard tenure-track careers. At best, the digital humanities provides the "meeting place" for these fellow-travellers. As Julia Flanders remarks,

Teaching digital humanists, then, proves challenging because instructors must design curricula that do not make strong presuppositions of any form of domain knowledge. The best curricula allow for emergent collaborations among practitioners, benefiting from and perhaps also relying on the existence of complimentary strengths among students. Yet instructors must be careful not to presuppose any central vantage point. Inevitably, a curriculum will prove too slack or too demanding, too theoretical or too technical, too slanted toward one type of application or another. In such cases, teaching depends on the good will of the participants, who assist each other with learning concepts from other domains.

The lack of presuppositions in the "digital humanities" brings us to a hot button issue, namely, whether coding is necessary to the digital humanities. There is a debate in the digital humanities community between those who think all digital humanists should learn to code and those who argue that learning to program is not necessary and perhaps even harmful to scholarly productivity.^[2] The slogan "More hack, less yack!" sometimes gets thrown around, suggesting that digital humanities is primarily about building things rather than talking about them. As Bethany Nowviskie reminds us, however, the original sense of the "yack" in that playful phrase was not aimed against academic theoretical analysis, but at wasteful committee work that frequently encompasses digital projects in the academy Nowviskie 2014. The XQuery Institute set out to cross this purported divide between builders and sayers in the digital humanities with as little fanfare as possible. My guiding intuition is that a complementarity exists between coding and encoding–by learning to query your markup, you become a more proficient encoder. So learning even a bit of programming should prove helpful to those working on digital markup projects. Of course, establishing this complementarity is easier with some programming languages than others, which brings me to XQuery.

Making the Case for XQuery

I may as well state upfront that I regard XQuery as a fantastic language for digital humanists. If you are involved in marking up documents in XML, then learning XQuery will pay long-term dividends. I do have arguments for this bit of bravado. My reasons for lifting up XQuery as a programing language of particular interest to digital humanists are essentially three:

Let's take each of these points in turn.

First, XQuery fits the domain of digital humanists. Admittedly, I am focusing here on a particular area of the digital humanities, namely the domain of digital text editing and analysis. In that domain, however, XQuery proves a nearly perfect match to the needs of digital humanists.

If you scour the online communities related to digital humanities, you will repeatedly find conversations about which programming languages to learn. Predictably, the advice is all over the map. PHP is easy to learn, readily accessible, and the language of many popular projects in the digital humanities such as Omeka and Neatline. Javascript is another obvious choice given its ubiquity. Others recommend Python or Ruby. At the margins, you'll find the statistically-inclined recommending R. There are pluses and minuses to learning any of these languages. When you are working with XML, however, they all fall short. Inevitably, working with XML in these languages will require learning how to use packages to read XML and convert it to other formats.

Learning XQuery eliminates any impedance between data and code. There is no need to import any special packages to work with XML. Rather, you can proceed smoothly from teaching XML basics to showing how to navigate XML documents with XPath to querying XML with XQuery. You do not need to jump out of context to teach students about classes, objects, tables, or anything as awful-sounding as "shredding" XML documents or storing them as "blobs." XQuery makes it possible for students to become productive without having to learn as many computer science or software engineering concepts. A simple four or five line FLWOR expression can easily demonstrate the power of XQuery and provide a basis for students' tinkering and exploration.

Second, XQuery allows for "full-stack" web development. What I mean by this is that XQuery can be used in conjunction with native XML databases to develop end-to-end web applications. This contrasts with other popular approaches, which require aspiring developers to learn SQL for interacting with relational databases, maybe to understand a bit about object-orientation, and to grasp how to express their outputs in HTML, perhaps with the help of a templating language. By contrast, it's perfectly feasible to write simple interactive web applications just using XQuery and a native XML database. Eventually, students will need to learn how to incorporate CSS and JavaScript to improve the appearance and usability of their applications. But they can get things off the ground without learning anything more than XQuery.

A third advantage of XQuery is that it's easy to learn. You wouldn't think this is the case if you read questions on Stackoverflow. But my suspicion is that the majority of those who find XQuery perplexing or even execrable have not bothered to learn about functional programming and run headlong into brick walls like single-assignment. It is indeed tough for experienced programmers to switch from imperative, object-oriented programming to functional programming. A lot of the idioms and design-patterns are just different. By contrast, digital humanists are generally learning to program for the first time. They don't have to unlearn habits from imperative programming-languages. Could Edsger Dijkstra's dictum–"It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration" Dykstra 1975–actually put digital humanists at an advantage over experienced programmers when learning XQuery (substituting BASIC for any imperative or object-oriented language)?^[3] In a more serious vein, functional programming has the advantage of not allowing beginning programmers to "mess up" their data. So long as you stay away from extension functions, you can promise students that nothing they do can corrupt their data. This brings a certain measure of relief to new programmers, who can "safely" explore the standard function library.

XQuery is fairly simple and straightforward at heart. While I wouldn't recommend that beginners tackle the XPath and XQuery Functions and Operators 3.0 recommendation, as instructive as it is to experts, working through groups of related functions using a site like Priscilla Walmsely's FunctX XQuery Functions^[4] is a great method to familiarize beginners with XQuery expressions.

Here's a last point about XQuery vis-a-vis other functional programming languages. I used to refer to XQuery as "functional programming light" prior to Version 3.0. However, with support for higher-order functions in XQuery 3.0 and built-in functions mapping and folding, XQuery stacks up nicely against any other functional programming language out there. However, that doesn't mean that one should automatically jump into those deep waters with beginners.^[5] In general, functional programming relies heavily on recursion, which is notoriously difficult for non-programmers to master. By contrast, FLWOR expressions make it possible to write many expressions without using recursion.^[6] While I have taught recursion in all three contexts I discuss below, it's nice not to have to explain recursion too early in the learning process.

Three Case Studies

I've taught XQuery in three different contexts. Before describing my experiences, it's reasonable to ask what qualifies me to teach XQuery. My background is purely in the humanities. I was a philosophy major who eventually earned a Ph.D. in theology. But I also had the privilege of growing up around computers, following a progression from Applesoft Basic to Pascal to Visual Basic.

When I was hired as a curator in the Princeton Seminary Library, I was asked to develop a digital library program for our special collections. I started up a project in 2002 with C# and XSLT, which is still running today.^[7] But I had a difficult time implementing search functionality and began looking around for other options. In 2006, I attended a software conference (now sadly defunct) called Software Development West (SD West), where I signed up for sessions on XPath by Debbie Lapeyre and XQuery by Jason Hunter. Suffice it to say, the combination of these sessions persuaded me that we should switch out digital library platform from .NET to MarkLogic. The Princeton Theological Seminary Library purchased a license for a standard edition of the MarkLogic in June 2007. In July, our metadata librarian and I took a three day "Introduction to MarkLogic Server: Developer Course." We worked with MarkLogic Professional Services to rewrite our digital library platform from C# and XSLT to XQuery. Meanwhile, I spent the summer reading and re-reading Priscilla Walmsley's XQuery Walmsley 2007.

So much for my qualifications. Like many digital humanists, I pieced together what I needed to know as I needed to know it. I was at least a little farther ahead of the people I planned to teach.

Obstacles for Digital Humanists

What are the unique challenges of teaching XQuery to digital humanists? I think we can extrapolate at least three hurdles digital humanists face when seeking to learn XQuery.

The first is simply gaining a vision for the place of programming in their scholarship. Why make the effort to learn to program? Is that not a step outside the humanities into a foreign, and sometimes hostile, world? Is it not better to leave programming to the experts? The move from word processing to markup languages is far more incremental than the leap from encoding to coding. The current enthusiasm for digital humanities has perhaps lessened this line of concern. Still, the questions remain pertinent given the demands on scholars' time.

Teaching XQuery to digital humanists requires helping them envision how their new skills and insights will advance their scholarship. We cannot assume any intrinsic motivation to learn programming for its own sake. This renders the majority of the standard teaching materials for programmers problematic, at least as standalone documents. Most programming manuals assume that readers are already motived to learn a given technology and, after perhaps a few paragraphs about its power and usefulness, immediately jump into the details. For digital humanists, a better approach would be to explain general concepts with reference to concrete research problems. For instance, using XQuery to interact with RESTful APIs might be learned in the context of teaching digital humanists how to use the Alchemy API to carry out sentiment analysis on their texts.

A second major challenge is that most digital humanists encounter XQuery without any background in computer science or software engineering. This makes for humorous "namespace" conflicts with natural language. While teaching the XQuery Summer Institute, we began keeping an informal list of technical terms in XQuery that have natural language cognates: evaluate, expression, predicate, etc. Even technical slang like "grok" needs explanation. David Michelson played our "faculty Luddite" role during the institute.^[16] Whenever an unexplained term popped up, Michelson would request an explanation on behalf of the group. His role was important because, as technical terms piled up, participants tended to defer questions or pass over them altogether.

Of course, just defining terms only gets you so far. The primary issue was filling in the necessary background knowledge. Digital humanities can be more demanding than traditional computer science courses. Whereas there is a recommended sequence of courses in computer science programs, which introduce and then deepen a body of knowledge over time, digital humanities courses and training sessions generally presuppose little to nothing. Digital humanists must frequently tackle new concepts like typing and scoping while also learning new computational techniques.

A related issue is that few digital humanists have experience with software engineering. Training resources for XQuery frequently assume readers have experience with common tools like relational databases, making analogies to SQL and relational tables, for example, when explaining joins in XQuery. This assumption makes perfect sense when teaching technologists but it doesn't work for digital humanists since you'll inevitably need to explain how a relational database functions for them to get the analogy. Best practices like using source control and writing unit tests also require clarification.^[17] This problem becomes especially acute when digital humanists aspire to build systems for the project. Broadly speaking, the divide between librarians and faculty becomes wider here. Librarians frequently do need to develop systems and will thus need to master the myriad of software engineering practices and tools. Many faculty members will experience software engineering as a domain of diminishing returns. An effective digital humanities center or campus network will connect faculty to developers when they are ready to build production systems for their projects.

A third challenge is that digital humanists frequently need to push beyond the boundaries of XQuery for the purposes of their research. In our digital humanities class at Vanderbilt, for example, we wanted to reproduce some of the analytical techniques that Matthew Jockers discusses in Macroanalysis: Digital Methods and Literary History. As far as I am aware, there is no statistical library for XQuery. So we demonstrated how to use XQuery to gather data from documents and then how to import that data into R for statistical analysis. While the combination of XQuery and the R programming language is potentially extraordinarily powerful, switching contexts between the languages proved too difficult. We did not have sufficient time to introduce R, RStudio, ggplot2, etc. and the students who tried to incorporate this technique into their final projects were not successful. If XQuery had a data analysis library akin to Python's pandas,^[18] for instance, it would be easier for them to use XQuery to produce the kind of digital text analysis that many have in mind when contemplating computational applications. Recent work bridging XQuery and SPARQL is promising because a significant number of digital humanities projects include a network analysis component. See, e.g., Cagle 2011 Enriching the XQuery ecosystem with libraries to carry out basic "digital humanities" functionality would help to make the language more accessible. eXist leads the way in this respect with its package system, incorporating such useful packages as the Stanford Named Entity Recognizer.

Making XQuery More Accessible

As I wrap up, I'd like to suggest a few ways to improve the accessibility of XQuery for digital humanists.

First, we need to develop instructional materials for digital humanists. In general, the materials for learning XQuery are few compared to other programming languages, reflecting no doubt its level of usage in the general technology community. We may be thankful for Priscilla Walmsley's XQuery Walmsley 2007. Where would we be without her book? I hope that she and O'Reilly will consider a second edition now that XQuery 3.0 has reached recommendation status. We may also be grateful that Erik Siegel and Adam Retter are currently writing a book for O'Reilly on eXist; it's already an extremely valuable resource in its "rough cuts" format. Siegel and Retter 2014 Regrettably, Ron Hitchens' Getting Started With XQuery has gone out of print at the Pragmatic Bookshelf, presumably because new versions of eXist and MarkLogic have rendered dated its instructions for getting started. Hitchens 2008 Given that Hitchen's book was among the most accessible for beginners, a gap currently exists in the literature for introducing beginners to XQuery and XML databases. The excellent XQuery Wikibook provides a bunch of really useful examples of XQuery in action.^[19] What's needed is a textbook for use in digital humanities workshops and classrooms. Crucially, the sample applications in this proposed textbook should center on narrative documents and, for the most part, avoid mathematical examples. In other words, if you plan to teach recursion, build an algorithm to validate palindromes rather than solve the Fibonacci sequence.

Another way to improve the accessibility of XQuery would be to encourage the sharing of datasets and code. As mentioned, the students in our digital humanities seminar at Vanderbilt experienced difficulty finding open datasets in some fields of the humanities. The problem is especially acute in art history but there is a lot less TEI on GitHub than you might expect. (There are, of course, many notable exceptions.) Libraries and other cultural institutions that produce TEI should adopt a default policy of sharing their data under Creative Commons licenses. The same point applies, ceteris paribus, to sharing code. Sharing examples of XQuery on GitHub or just as Gists can really prove helpful to those getting started.^[20]

A final recommendation is more ambitious. My hope is to integrate XPath and XQuery into the teaching of TEI and other markup standards in the digital humanities community. As Jonathan Robie indicated during our XQuery Institute, encoding and coding should be complimentary, iterative activities. More strongly put, the full implications of digital encoding do not become apparent until scholars learn to reorder, remix, and rework their texts computationally. We all know the effort involved in marking up a text in TEI. If the end goal is simply to transform the marked up document into HTML or PDF for publication on the web, then we cannot expect scholars to continue to make that investment of time over the long run. By providing scholars with the computational tools to analyze their documents, we repay their efforts to encode them. Moreover, teaching XQuery alongside TEI would benefit scholars' understanding of markup, allowing them deepened insight into why particular encoding decisions might be more or less efficient for particular purposes.

What would this new method of teaching TEI look like? The syllabus has yet to be worked out. However, I imagine that the basics of the approach could be accomplished in three day workshops, which aimed less at building systems with XQuery than at providing a sufficient introduction to the language for digital humanists to commence an iterative process of encoding and querying. Where they go from there would be up to them–as with any programming language, there are no limits on where XQuery can take them.

Conclusion

The pressures on XML as a markup standard in the general community of technologists also exist in the digital humanities, though in more muted fashion. While most digital humanists working on textual projects continue to work in XML, they interact with librarians and other technologists who have increasingly turned to JSON to transfer data across the wire. From my perspective, the digital humanists would be impoverished if leaders in the digital humanities community sought to rewrite TEI in JSON. What would that project look like? More likely, digital humanists might just give up on marking up documents in favor of conducting statistical analyses on datasets of unstructured texts. If we wish to avoid this future and preserve a place in the digital humanities for the kind of critical editions that XML markup affords, it behooves us to assist scholars with climbing up the XML value chain–from mastering XML languages like TEI to learning analytical technologies like XPath and XQuery.