Balisage Paper: Multi-channel eBook production as a function of diverse target device capabilities
Eric Freese
Director/Solutions Architect
Aptara
Eric Freese is a veteran of publishing production optimization. From the early days
of SGML, he has worked in roles as varied as consultant, software developer, content
architect and semantic web technologist in industries including technical publishing,
defense, commercial software and legal publishing. In his role as Director/Solutions
Architect, Eric helps Aptara customers efficiently transition to cost-saving digital
publishing models that simultaneously support eBook and other electronic delivery
platforms, as well as traditional print production.
Copyright © 2010 by the author. Used with
permission.
Abstract
Publishers have a history of managing data for the long-term. In the past this meant
warehouses of proof sheets. As the publishing process became electronic, assets were
archived on tape or disk in the event of a need to re-release (perhaps due to a popular
subsequent movie or other event). However, the challenge shifted from storage of
materials to file compatibility and management of software versions. Then along came
XML, which offered a vendor-agnostic format in which to, at a minimum, archive data
if not becoming the format in which the data was authored. Now that publishers find
themselves in the midst of an eBook boom they find themselves, yet again, dealing
with managing different formats and, to complicate matters, dealing with different
implementations of the same standard. This paper will discuss the current state of
the trade, textbook, and journal publishing industry as it pertains to creating eBooks
for current and backlist resources. It will also discuss the challenges faced by
publishers as the technology continues to move forward toward “enhanced eBooks” at
a pace faster than the standards can be updated.
A sample project will be presented which presents the challenges of producing eBooks
in the current environment as well as demonstrates how current standards might support
the next generation of eBooks.
Table of Contents
- Introduction
- eBooks Explained
-
- Leading eBook Formats
-
- EPUB
- AZW/MOBI
- PDF
- Survey of Devices
-
- Amazon Kindle
- Barnes & Noble nook
- Sony Reader
- Apple iPod Touch/iPhone
- Apple iPad
- T-mobile G1 (HTC Dream)
- enTourage eDGe
- OLPC XO
- Enhanced eBooks
- The Project
-
- Text Conversion (aka Overlap Happened)
- Insertion of Illustrations
- Creation of Hidden Text
- Insertion of External Links
-
- Sidetrack: Device Specific Builds and Graceful Degradation
- Media Links
- Future work
- Conclusion
eBooks Explained
Ebooks really aren't anything new. Project Gutenberg was started in 1971 with a goal
to digitize and archive cultural works and to encourage the creation and distribution
of eBooks. Designs for general purpose portable reading devices where first developed
at PARC in the 1970s. In 1998 the first commercial eBook readers were made available.
In 1999, the first version of the Open eBook Publication Structure (OEBPS) was released
with 3 goals:
-
to give content providers (e.g., publishers, and others who have content to be displayed)
and tool providers minimal and common guidelines which ensure fidelity, accuracy,
accessibility, and presentation of electronic content over various electronic book
platforms.
-
to reflect established content format standards.
-
to provide the purveyors of electronic-book content (publishers, agents, authors et
al.) a format for use in providing content to multiple reading systems.
Since then, many document formats have been introduced by many software vendors.
Also even more devices were developed and marketed. But it wasn't until 2007, when
Amazon launched the Kindle, that eBooks and eBook readers seemed to capture the consumer's
imagination. In the last couple of years, the industry has grown and matured at a
very rapid pace. While there is still a plethora of file formats, three have become
the leading formats: EPUB, AZW, and PDF.
Leading eBook Formats
EPUB
The EPUB standard, developed by the International Digital Publishing forum (IDPF)
is actually a combination of 3 standards:
-
OEBPS Container Format (OCF) - describes the general-purpose container technology
in the context of encapsulating OEBPS publications and OPTIONAL alternate renditions
thereof
-
Open Packaging Format (OPF) - defines the mechanism by which the various components
of an OPS publication are tied together and provides additional structure and semantics
to the electronic publication
-
Open Publication Structure (OPS) – the current version of OEBPS
EPUB files allow the text to be reflowed to allow for different font and sizes and
screen sizes and orientations. As a user changes the font size or screen orientation,
the data in the screen is redrawn to accommodate the request.
An EPUB file can essentially be thought of as a "web site in a box". In most cases,
an EPUB file (which is a zipped file) consists of a set of XML (usually XHTML) files
plus any contained image or media files and CSS stylesheets. There is also a set
of files defined within the EPUB standard that must also reside within the EPUB file.
The data within the EPUB file provide the information needed to present the data on
an eReader device.
AZW/MOBI
In 2007, when Amazon launched the Kindle eBook reader, they created the AZW format.
It is based on the Mobipocket standard, with a slightly different serial number scheme
and its own Digital Rights Management (DRM) formatting. It is perhaps somewhat ironic
that the Mobipocket e-book format is based on the OEBPS standard. It uses XHTML and
can include JavaScript and frames. It also supports native SQL queries to be used
with embedded databases. Notably, Eastern European letters with diacritical marks
and Pacific rim characters are not supported. AZW files can also be reflowed, just
as EPUB files can be.
PDF
The Portable Document Format was created by Adobe Systems, initially to provide a
standard form for storing and editing printed publishable documents. Because PDF documents
can easily be viewed and printed by users on a variety of computer platforms, they
are very common on the World Wide Web. The specification of the format is available
without charge from Adobe.
PDF files are able to embed fonts, images, and other documents, making them well-suited
for eBook delivery. A PDF file contains one or more zoomable page images.
The format is designed to reproduce page images, therefore text traditionally could
not be re-flowed to fit the screen width or size. As a result PDF files designed for
printing on standard paper sizes are less easily viewed on screens with limited size
or resolution, such as those found on mobile phones and PDAs. Adobe has addressed
this by adding a re-flow facility to its Acrobat Reader software, but for this to
work the document must be marked for re-flowing at creation, which means existing
PDF documents will not benefit unless they are tagged and resaved.
Survey of Devices
There are many different eBook reader devices available and seemingly more every day.
These devices can be divided into 2 basic classes. The first is based on an E Ink
electronic paper display. These displays can currently only show shades of gray (although
research is being done on color E Ink), are viewable in direct sunlight, and require
no power to maintain a static image. There is no back lighting, which reduces eye
strain, but makes them unreadable in the dark. The Amazon Kindle and Sony Reader are
examples of this class of reader. Many within this class of reader can play MP3 and
other types of audio files. The second class uses backlit LCD screens and are often
more than dedicated eBook reading devices. For this project 8 devices were included
in the test set. These devices come from both classes and provide a mix of different
features and capabilities.
Amazon Kindle
Amazon Kindle is a software and hardware platform developed by Amazon.com for the
rendering and displaying of e-books and other digital media. It was one of the first
widely accepted eBook readers. The Kindle supports several ebook file formats including
AZW/MOBI and PDF. The first version of the device was first sold in the U.S. in 2007.
The Kindle hardware device uses an E Ink brand electronic paper display that features
16 shades of gray and a six inch E Ink display. It downloads content over Amazon's
Whispernet. The Kindle hardware device is used without a computer connection, and
Amazon Whispernet is accessible without any monthly fee or wireless subscription.
Because the eBooks bought on the Kindle are delivered over its wireless system, the
user does not see the AZW files during the download process.
Kindle software applications exist for Windows, iOS, BlackBerry, Android and Mac OS
X. This allows Kindle content to be read on devices running any of these operating
systems. In software updates released in June 2010, capabilities were added to the
Kindle software applications that are not supported by the Kindle device, most noteably
the inclusion of audio and video. In doing so, Amazon created a requirement for graceful
degradation within eBook files. Through a technology called "Whispersync," customers
can connect reading progress, bookmarks, and other information across Kindle hardware
devices and other mobile devices.
Amazon's use of a proprietary format has allowed them to optimize the reader software
to work extremely well with their data format. The Kindle also has access to perhaps
the largest online bookstore, as well as various independent sources and free content
providers. Also, for some time, Amazon was seen as the industry leader and therefore
able to dictate to publishers what the prices on eBooks should be. With the advent
of the Apple iPad, Amazon has been working to add more online capabilities to the
Kindle device, such as an improved browser and social networking applications such
as Facebook and Twitter.
Barnes & Noble nook
The nook is an eBook reader developed by Barnes & Noble and was one of the first eBook
readers based on the Android platform. The nook supports several eBook file formats
including EPUB and PDF. The device was first sold in the U.S. 2009. The nook includes
Wi-Fi and 3G wireless connectivity, a six inch E Ink display, and a separate, smaller
color touchscreen that serves as the primary input device and controls much of the
navigation within the reading device itself..
One differentiating feature of the nook is the "LendMe" feature: some books are licensed
by their publishers for sharing. In those cases, the purchaser is permitted to share
a book once, with one other user, for up to two weeks. Barnes & Noble also developed
reader application software for iOS platforms, Mac, and Windows.
This device utilizes Adobe DRM for protected files. The DRM rules allow any purchased
e-book to be read on up to six devices, at least one of which must be a personal computer
running Windows or Mac OS X.
Sony Reader
The Sony Reader is an e-book reader manufactured by Sony. It can display PDF and EPUB
format, as well as personal documents, blogs, RSS newsfeeds, JPEGs, and Sony's proprietary
BBeB ("BroadBand eBook") format, although Sony has converted all the books in their
online bookstore to the EPUB format, seemingly deprecating the BBeB format. The Reader
is usable in portrait or landscape orientation. The reader uses an iTunes Store-like
interface to purchase books from Sony Connect eBook store. The PRS-600 (Touch Edition)
model used in this project can display 8-levels of grayscale and has a six inch E
Ink display.
This device also utilizes Adobe DRM for protected files.
Apple iPod Touch/iPhone
The iPod Touch is a portable media player, personal digital assistant, and Wi-Fi mobile
platform designed and marketed by Apple Inc. The device features a color multi-touch
graphical user interface on a 3.5 inch LED backlit display. It allows wireless access
to the iTunes Store, and also has access to Apple's App Store, enabling content to
be purchased and downloaded directly on the device. Apple Inc. has sold more than
20 million iPod Touch units.
The iPod Touch and the iPhone, a smartphone by Apple, share the same hardware platform
and run the same iOS operating system. The iPod Touch lacks some of the iPhone's features
and associated apps, such as access to cellular networks and a built-in camera (and
microphone on older models). As a result, the iPod Touch is slimmer and lighter than
the iPhone. Multimedia, which is available as a single "iPod" app on the iPhone, is
split into music and movies on the iPod Touch.
While not originally marketed as an eBook reader devices, Apple changed directions
in June 2010 when they released their iBooks app (using Apple's Fairplay DRM) to these
devices as part of the iOS4 release. In addition, several apps have been developed
which allow eBooks to be read on the Touch including Stanza which is able to read
many formats including AZW, EPUB and PDF as long as there is no DRM present. With
number of units sold, some have called it the mostly widely sold eBook reading device,
although it is impossible to say how many users actually use it as such.
Apple iPad
The iPad is a tablet computer developed by Apple Inc. Similar in functionality to
the iPhone or iPod Touch, it currently runs the 3.2 version of iOS, with a user interface
redesigned to take advantage of the larger screen. The iPad can display EPUB files
natively and uses Apple's Fairplay DRM scheme. The iPad has a 9.7-inch LED backlit
multi-touch display.
As Apple's first device to use its iBookstore service and companion iBooks eBook reading
application, the iPad has been compared to the Kindle and the nook. Users can load
their own EPUB files by loading them into iTunes and then syncing to the device.
Amazon has released a Kindle for iPad app. Barnes and Noble, Kobo, and Borders also
provide apps that provide access to their eBooks as well. This range of readers makes
the iPad a common platform on which eBooks from different vendors and different formats
can be read.
T-mobile G1 (HTC Dream)
The G1 is an Internet-enabled 3G smartphone with hardware designed by HTC. It was
the first phone to the market that uses the Android mobile device platform. It features
a 3.2 inch color touchscreen. Like the iPhone is was not originally marketed as an
eBook reader, but several apps have also been developed for it including Aldiko, FBReader
and WordPlayer. These apps can all read non-DRM EPUB and have various features such
as night-time reading (white text on black screen) and other preference-base customizations.
enTourage eDGe
The eDGe is a “dualbook” which combines the functions of an e-reader, netbook, notepad,
and audio/video recorder and player into a single unit. The eDGe has a clamshell design
(that can flip to be used as a book or as tablet) and two displays, a 9.7-inch E Ink
display and a 10.1-inch LCD. The device runs Android software which controls both
screens. The E Ink display is used to display eBooks and also provides an onscreen
note-taking/drawing capability. This feature also allows the user to take notes and
make annotations and save the markings as PDF.
The reader software within the eDGe can display both EPUB and PDF. This device also
utilizes Adobe DRM for protected files.
OLPC XO
The OLPC XO-1 is an inexpensive subnotebook computer intended to be distributed to
children in developing countries around the world, to provide them with access to
knowledge, and opportunities to "explore, experiment and express themselves" (constructionist
learning).
The subnotebooks are designed for sale to government-education systems which then
give each primary school child their own laptop. These rugged, low-power computers
use flash memory instead of a hard drive, and come with a distribution of Linux derived
from Red Hat's Fedora. Mobile ad-hoc networking via 802.11s WiFi mesh networking protocol
is used to allow many machines to share Internet access as long as at least one of
them can see and connect to a router or other access point. The first-generation OLPC
laptops have a novel low-cost LCD screen. This screen has teh abiity to switch modes.
When lit primarily from the rear with the white LED backlight, the display shows a
color image composed of both RGB and grayscale information. When lit primarily from
the front by ambient light, for example from the sun, the display shows a monochromatic
(aka black and white) image composed of just the grayscale information.
Applications on the XO are called activities. One of the activites that is available
is the Read activity which can read EPUB and PDF files. The ability to use this device
as a reader in any environment made it an interesting possibility for this project.
Enhanced eBooks
The term “enhanced” eBooks has been used to mean many different things. For example,
David Baldacci released his book “Deliver Us from Evil” as an enhanced eBook. In
this case, in addition to the story, he included behind the scenes looks at how a
book is written including audio, video, photos and an alternate ending, much like
the special features found on many DVDs.
Essentially an enhanced eBook is a combination of media and book content that utilizes
any mixture of text, audio, still images, animation, video, and interactivity content
forms. The formats used to create a literary fiction eBook can include the addition
of audio-visual elements and interactive contents allowing new form of creativity.
The user (e.g., reader) might have an opportunity to participate in events occurring
to characters, to feel influence of a musical part of a narration and graphic part.
The Project
The goal of the project was to develop an eBook that demonstrates a number of enhanced
eBook capabilities, but still works on a standard eBook reader, such as those mentioned
above, to the greatest extent possible. The functionality to be attempted includes:
-
substantial table of contents - tests folding and other navigation capabilities
-
footnotes - test navigation within the eBook and presentation. Two different methods
were evaluated: bi-directional footnotes and embedded footnotes.
-
illustrations – test graphics capabilities and memory management
-
creation of hidden text – tests CSS support and search capabilities
-
external links – tests support of linking to web pages outside of the eBook
-
media links – test support of audio and video
-
interactivity - test methods for including interactive content within the eBook
Another requirement of the project was that it should include public domain or freely
available source material. This allows the final product to be freely distributed
without concern for licensing issues.
The World English Bible was selected as the source text. The text is available in
XML from the World English Bible website (http://worldenglishbible.org) in the Open Scriptural Information Standard (OSIS) format. This version of the
Bible also includes footnotes throughout along with cross reference between verses.
Images are available from the LaVista, Nebraska, Church of Christ website (http://www.lavistachurchofchrist.org/Picture.htm). The staff of the church have scanned images from several books that are now in
the public domain and have made them available. Many of these images were downloaded
for insertion into the eBook file. MP3 recordings of each chapter are available from
Audio Treasure (http://www.audiotreasure.com/webindex.htm). External links include place names shown on Google Maps. KML files that connect
verses to locations are available from the Bible Geocoding project (http://www.openbible.info/geo). Links to video lessons are also available at theBible.net (http://www.thebible.net/video/).
Text Conversion (aka Overlap Happened)
The text conversion from OSIS XML to XHTML for inclusion in the eBook file appeared
to be fairly straightforward at first glance. However, further analysis revealed
that there was a minimal amount of structure contained within the file. Also, the
entire Bible was contained within a single file. The snippet below shows many of the
markup problems:
1 <p>
2 <verse eID="Luke.20.47"/>
3 <chapter eID="Luke.20"/>
4 <chapter sID="Luke.21" osisID="Luke.21"/>
5 <verse sID="Luke.21.1" osisID="Luke.21.1"/>He looked up, and saw the rich people
6 who were putting their gifts into the treasury.<verse eID="Luke.21.1"/>
7 <verse sID="Luke.21.2" osisID="Luke.21.2"/>He saw a certain poor widow casting in
8 <milestone type="x-noteStartAnchor"/>two small brass coins.
9 <note type="translation">literally, “two lepta.” 2 lepta was about 1% of a day’s wages
10 for an agricultural laborer.</note><verse eID="Luke.21.2"/>
11 <verse sID="Luke.21.3" osisID="Luke.21.3"/>He said,
12 <q sID="Luke.21.3.149" who="Jesus" n="" type="x-doNotGeneratePunctuation"/>“Truly I
13 tell you, this poor widow put in more than all of them,<verse eID="Luke.21.3"/>
14 <verse sID="Luke.21.4" osisID="Luke.21.4"/>for all these put in gifts for God from their
15 abundance, but she, out of her poverty, put in all that she had to live on.”
16 <q eID="Luke.21.3.149" n=""/>
17 </p>
Within this <p> element, we see <verse> and <chapter> elements. However, these tags
are simply empty boundary markers, denoted by the sID (e.g. line 5) and eID (e.g.
line 6) attributes. Notice the verse end (line 2) and the chapter end (line 3) and
beginning (line 4) at the start of the <p>. The <milestone> element (line 8) marks
the beginning of the anchor text, but the end of the text is identified by the <note>
start tag (line 9). In many cases, <note> elements do not have <milestone> elements
to identify any text. Also notice the <q> (quote) elements. The starting indicator
(line 12) occurs in one “verse” and the ending indicator (line 16) occurs within a
different “verse”.
These overlapping structures presented a challenge in converting the data to XHTML.
It was desired that verse numbers be inserted into the text in order to allow readers
to more easily navigate the text. Since the primary use of the data is for human reading,
there was not a need to specifically delineate the verse boundaries, as this could
be determined programmatically later, if needed. Also paragraphs would be kept intact
as much as possible.
The process of converting the OSIS markup took several steps. First of all, an XSLT
stylesheet was written that attempted to convert the OSIS markup into nested XHTML
structure. The stylesheet processed the beginning and ending marker elements and
converted them to character sequences within the output text. The stylesheet also
created the separate files for each chapter. Since output from the stylesheet needed
to be well-formed in order to be processed further by other stylesheets, angle brackets
were replaced with "~~" to delineate markup. Some markup was also removed including
the verse ends.
<p>~~ENDchapter~~~~chapter osisID="Luke.21" sID="Luke.21"~~<verse osisID="Luke.21.1"
sID="Luke.21.1"/>He looked up, and saw the rich people who were putting their gifts into the
treasury.<verse osisID="Luke.21.2" sID="Luke.21.2"/>He saw a certain poor widow casting in
<milestone type="x-noteStartAnchor"/>two small brass coins.<note type="translation"
xid="w1aab1b7b5d709c14"> literally, “two lepta.” 2 lepta was about 1% of a day’s wages for an
agricultural laborer.</note><verse osisID="Luke.21.3" sID="Luke.21.3"/> He said, <q n=""
sID="Luke.21.3.149" type="x-doNotGeneratePunctuation" who="Jesus"/>“Truly I tell you, this poor
widow put in more than all of them, <verse osisID="Luke.21.4" sID="Luke.21.4"/>for all these
put in gifts for God from their abundance, but she, out of her poverty, put in all that she had to
live on.”<q eID="Luke.21.3.149" n=""/></p>
The data above shows the output of the original snippet after it had been transformed
by the first XSLT stylesheet.
Next a series of sed commands was run on the file to convert the character sequences
into XML tags. For example the "~~ENDchapter~~" sequence was changed to </chapter>.
Other sed commands were used to arrange the tags into the correct nesting order in
order to be well-formed (e.g. moving </chapter> tags before <p> tags. The sed tool
was used as a quick and dirty method for manipulating the overlapping markup in order
to prepare the data for further XML processing.
<chapter osisID="Luke.21" sID="Luke.21">
<p>
<verse osisID="Luke.21.1" sID="Luke.21.1"/>He looked up, and saw the rich people
who were putting their gifts into the treasury.
<verse osisID="Luke.21.2" sID="Luke.21.2"/>He saw a certain poor widow casting
in <milestone type="x-noteStartAnchor"/>two small brass coins.
<note type="translation" xid="w1aab1b7b5d709c14">literally, “two lepta.” 2 lepta
was about 1% of a day’s wages for an agricultural laborer.</note>
<verse osisID="Luke.21.3" sID="Luke.21.3"/>He said,
<q n="" sID="Luke.21.3.149" type="x-doNotGeneratePunctuation" who="Jesus"/>
“Truly I tell you, this poor widow put in more than all of them,
<verse osisID="Luke.21.4" sID="Luke.21.4"/>for all these put in gifts for God
from their abundance, but she, out of her poverty, put in all that she had to
live on.”
<q eID="Luke.21.3.149" n=""/>
</p>
...
</chapter>
The data above show the data after it has been provcessed by the sed batch commands.
The <chapter> markup now surrounds the <p> and <verse> markup now occurs within the
<p>. However, the overlapping <q> markup still exists.
Finally another XSLT script took the well-formed XML source file to prepare the files
needed to build the EPUB collection. Spans with class attributes were also added into
the markup to allow CSS formatting of the text. For example the span with the “jesus”
class (formerly <q> markup) allows for red-letter marking of the words of Jesus on
eBook readers that display color. Each book, chapter and verse were also given IDs
to provide anchor points for linking (e.g. footnote returns). Since milestones do
not occur consistently, it was decided to drop them and simply insert a marker indicating
that a footnote is present. In cases where quotes span over several paragraphs or
chapters, the quotes are closed at the end of the larger structure and restarted in
the preceding one. For example, the quote was closed at the end of each verse and
reopened after the next verse number markup had been inserted. This prevents the
verse numbers from being displayed in red. The single XML was divided into individual
files, each containing one chapter.
Two different methods were considered for footnotes. In the first method, footnotes
would be displayed separately from the text, but be linked via a footnote marker.
The footnotes were extracted from the source XML and stored in a separate file with
bi-directional links from the source to the footnotes and from the footnotes back
to the referring source. The markup shown below is an example of the final product
of the source text and its associated footnote.
<div class="chapter">
<span id="Luke.21"/>
<h2 class="chapter-title">Chapter 21</h2>
<p>
<span id="Luke.21.1" class="verse">1</span>
He looked up, and saw the rich people who were putting their gifts into the
treasury.
<span id="Luke.21.2" class="verse">2</span>
He saw a certain poor widow casting in two small brass coins.
<span id="fnref-w1aab1b7b5d709c14">
<a href="../footnotes.xml#fn-w1aab1b7b5d709c14" class="footnote"
title="Footnote Luke.21.2">ƒ</a>
</span>
<span id="Luke.21.3" class="verse">3</span>
He said, <span class="jesus">“Truly I tell you, this poor widow put in more
than all of them,</span>
<span id="Luke.21.4" class="verse">4</span>
<span class="jesus">for all these put in gifts for God from their abundance, but
she, out of her poverty, put in all that she had to live on.”</span>
</p>
</div>
<div class="footnotes">
<h3 class="chapter-title">Footnotes</h3>
...
<div id="fn-w1aab1b7b5d709c14">
<p><a href="Luke/Luke.21.xml#Luke.21.2">Luke.21.2 ⇑ </a>literally, “two lepta.”
2 lepta was about 1% of a day’s wages for an agricultural laborer.</p>
</div>
...
</div>
One problem that was encountered in using this format was the tendency of human readers
to hit the "previous page" button, thinking it was a back button. This would turn
the previous page in the footnotes section of the book, rather than returning back
to the original text.
In the second footnote scenario, embedded footnotes were used to keep the information
together and reduce the need for jumping back and forth (and confusion about the use
of the "previous page" functionality. In this case the footnote appears on screen
with the text, but is differentiated by a different color and spacing.
<div class="chapter">
<span id="Luke.21"/>
<h2 class="chapter-title">Chapter 21</h2>
<p class="text">
<span id="Luke.21.1" class="verse">1</span>
He looked up, and saw the rich people who were putting their gifts into the
treasury.
<span id="Luke.21.2" class="verse">2</span>
He saw a certain poor widow casting in two small brass coins.
<span class="notes"><span class="noteverse">2</span>literally, “two lepta.”
2 lepta was about 1% of a day’s wages for an agricultural laborer.
</span>
<span id="Luke.21.3" class="verse">3</span>
He said, <span class="jesus">“Truly I tell you, this poor widow put in more
than all of them,</span>
<span id="Luke.21.4" class="verse">4</span>
<span class="jesus">for all these put in gifts for God from their abundance, but
she, out of her poverty, put in all that she had to live on.”</span>
</p>
</div>
This method worked well on color displays, but required some extra styling in order
to work well on grayscale devices.
In addition to the preparation of the source text, the final stylesheet also generated
several metadata files needed to construct a valid EPUB file. This includes the OSF
file which contains the Dublin Core metadata, the file manifest and the spine (which
lists the files in the order in which they appear). An NCX file is also generated
which is used by the eBook reader to present the table of contents. Some readers
are able to present collapsible TOCs, so it is important to nest the entries in order
to allow that functionality when possible.
Insertion of Illustrations
Artists through the ages have used the Bible as a source for subject matter. Since
the source text contains no illustrations at all, a method was developed to insert
graphic images at the correct places within the text. As stated previously, most
of the source material (approximately 350 images) was found on a website of the LaVista
Church of Christ. These images are mostly black and white which will display well
on the E Ink screens of the devices.
An XML markup scheme was developed to allow an XSLT stylesheet to insert the graphic
links at appropriate places. An example of the insertion markup is shown below as
is a sample of the resulting image markup in the text. The stylesheet also added
the graphics to the manifest information within the OSF file.
<links>
<image verse="Luke 16.21" file="Luke.16.21.jpg" caption="Lazarus and the Rich Man"/>
<image verse="Luke.22.61" file="Luke.22.61.jpg" caption="Peter's Denial"/>
<image verse="Luke.23.5" file="Luke.23.5.jpg" caption="Jesus Before Pontius Pilate"/>
<image verse="Luke.23.23" file="Luke.23.23.jpg" caption="Jesus Beaten"/>
</links>
<br/><span>
<img src="../images/Luke.16.21.jpg" alt="Lazarus and the Rich Man"/>
</span><br/>
Creation of Hidden Text
In most eBook readers, when searching capability is provided, it only indexes the
PCDATA text, not attribute values. In order to make navigation of the Bible file
easier, it was suggested that hidden text be added at each book, chapter, and verse
containing alternate search strings that would allow users to go directly to any anchor
point. This hidden text would also allow alternate spellings and abbreviations to
be used, making navigation even more intuitive. For example, a reference to Paul's
second letter to Timothy, could be entered by a user as “2 Timothy”, "2nd Timothy”,
“II Timothy”, “2 Tim”, “2nd Tim”, “II Tim”, etc. These different variations are added
to each anchor point in a <span> element with an class attribute set to “hidden”.
The CSS stylesheet then attempts to hide the text, while making it searchable. Our
Luke example with the hidden text is shown below.
<div class="chapter">
<span id="Luke.21"/>
<h2 class="chapter-title">Chapter 21</h2>
<p>
<span id="Luke.21.1" class="verse">1</span>
<span class="hidden">Luke 21:1</span> He looked up, and saw the rich people who
were putting their gifts into the treasury.
<span id="Luke.21.2" class="verse">2</span>
<span class="hidden">Luke 21:2</span> He saw a certain poor widow casting in two
small brass coins.<span id="fnref-w1aab1b7b5d709c14">
<span id="fnref-w1aab1b7b5d709c14">
<a href="../footnotes.xml#fn-w1aab1b7b5d709c14" class="footnote"
title="Footnote Luke.21.2">ƒ</a>
</span>
<span id="Luke.21.3" class="verse">3</span>
<span class="hidden">Luke 21:3</span> He said, <span class="jesus">“Truly I
tell you, this poor widow put in more than all of them,</span>
<span id="Luke.21.4" class="verse">4</span>
<span class="hidden">Luke 21:4</span>
<span class="jesus">for all these put in gifts for God from their abundance, but
she, out of her poverty, put in all that she had to live on.”</span>
</p>
</div>
This intentionally introduced redundancy met with various degrees of success. Some
devices didn't support the “display:none” directive in the CSS file and displayed
the text. Some didn't display the text, but also didn't index it and others performed
as had been hoped.
Insertion of External Links
The ability to link to outside materials is seen as a “must-have” for enhanced eBooks.
In designing this functionality, it was already known that most of the first generation
readers (nook, Kindle, Sony) would not be able to perform this task. Therefore, a
new facet of the project was introduced: the ability to specify which features to
include in an eBook and which to skip. More on that below in the sidetrack.
Geographical names occur throughout the Bible. Many places mentioned either no longer
exist or are known by different names, making it difficult for a reader to know where
an even might have occurred. The Bible Geocoding project uses information from several
reference sources to build KML files for use in Google Maps and Google Earth. The
snippet below shows the information contained within the KML files for two locations
mentioned in the book of Luke.
<Placemark>
<name>Abilene</name>
<description><a href="http://www.gnpcb.org/esv/search/?q=Luke+3%3A1">Luke 3:1
</a></description>
<styleUrl>#obi-normal</styleUrl>
<Point>
<coordinates>36.091710,33.587300</coordinates>
</Point>
</Placemark>
<Placemark>
<name>Arimathea</name>
<description><a href="http://www.gnpcb.org/esv/search/?q=Luke+23%3A50">Luke 23:50
</a></description>
<styleUrl>#obi-normal</styleUrl>
<Point>
<coordinates>35.180162,31.832739</coordinates>
</Point>
</Placemark>
An XSLT stylesheet was developed that would convert the KML into a form that would
be easier for the insertion mechanism to process. This stylesheet also reversed the
order of the coordinates to work in the URL call to Google Maps. The XML scheme developed
for the geographic points resembles that for the image insertion.
<links>
<geo verse="Luke.3.1" point="33.587300,36.091710" name="Abilene"/>
<geo verse="Luke.23.50" point="31.832739,35.180162" name="Arimathea"/>
</links>
The links to Google Maps are built on the place name within the flowing text. Another
XSLT stylesheet uses the verse attributes shown above to locate the starting point
from where to search in the text. Recall that the only thing marking the beginning
of a verse is the <span class='verse'> markup. Once the appropriate starting point
is located in the tree, each text node until the following verse marker is examined
to determine if it contains the value in the name attribute. If so, an HTML <a> element
is wrapped around the text pointed to a coordinate in Google Maps. When the link
is activated, a web browser can be opened and the page displayed.
As expected, this functionality did not work in the first generation eBook readers
since they have limited browser capabilities. However, in the others, there were
various degrees of success. On the iPad, the user is asked if they would like to
leave the iBooks app to open the browser. On the eDGe, the map opens on the LCD screen,
as expected.
Sidetrack: Device Specific Builds and Graceful Degradation
As mentioned in the previous section, the hidden text worked differently in different
devices. Therefore, it too, might be a candidate for the include/don't include list.
As this concept was considered further, it seemed that almost any of the additional
functionality beyond the straight text could be managed in this fashion. This would
also allow differences in functionality to be recognized and items inserted based
on the target device. Another example of this might be to have colorized versions
of the graphics for the readers with color screens and black and white for the gray
scale displays. While device specific builds allow the production of EPUB files that
are more efficient with space by not including data that a device cannot use, it also
increases the complexity of distribution. For example, the devices that support the
Adobe DRM can interchange eBooks between them, but have different capabilities. It
would not make sense to add capabilities for the eDGe to a file that was being distributed
on the Barnes & Noble site, since most buyers would be plannig to use the nook.
In addition, Amazon has also added to the debate. When they updated the platform-specific
Kindle software applications in June, they introduced capabilities that are not supported
on the current Kindle devices. They do not allow publishers to upload different versions
of the the files to the Amozon website. This forces publishers to adopt a strategy
of graceful degradation within books in which they want to add audio and video capabilites.
Graceful degradation essentially means that there is markup that identifies enhanced
capabilities and the devices or software must know to either attempto handle the markup
or ignore it. If the markup is to be ignored, information can be provided to let
the reader know that they might be missing something.
As more eBooks are enhanced with media and other capabilities, this debate will continue
to occur.
Media Links
Another hallmark of enhanced eBooks is the use of multimedia resources to improve
the reading experience. Many people believe that EPUB does not support media, which
is simply wrong. The EPUB supports inclusion of different media types and can select
the appropriate player based on the mimtype given to a particular file. The limitation
is most often with the reading devices themselves. The Kindle device does not include
codecs for video, while the iPad cannot run Flash. Both of these can be embedded into
an EPUB file..
There is some debate in the eBook community about the best way to include media files
in EPUB files. There are essentially two options: 1) include the files in the EPUB
file itself; 2) reference the resources via HTML <a> tags. Each option has its benefits
and drawbacks.
The inclusion of the files in the EPUB file assures that the resource is available,
whether the eBook reader is online or not. It also reduces the reliance on a web
browser to retrieve the information. It is also faster since there is not delay while
waiting to download. The chief drawback is the impact this has on the EPUB file itself.
As mentioned earlier, the EPUB file is essentially a zip file with a specified set
up internal files and a “.epub” file suffix. Most audio and video formats are already
compressed, so therefore they do not compress significantly inside the EPUB file.
This requires memory overhead to open the files inside the readers. It also requires
longer download times for the books when they are first loaded onto the eBook reader.
Referencing files produces smaller EPUB files, but requires a mechanism (usually a
web browser) to retrieve and present the resources. This requires that the eBook
reader be online. There will also be a lag between the time the link is selected
and the resource is downloaded. This is especially noticeable with large video files.