Balisage Paper: Easy XML Serialization of C# and Java Objects

Introduction

Object serialization is the process of rendering an object into a state that can be stored persistently. Serializing objects to XML brings many advantages over binary encoding because XML is human and computer readable, and it also aims to have better interoperability between different programming languages. Interoperability is an important issue in distributed object-based systems, because it allows the communication of programs (clients and servers) written in different object-oriented programming languages. There are some fundamental issues that have to be agreed by the different programming languages to be able to reach interoperability. Some of these are related to the data type mapping, object representation, messages, serialization and de-serialization.

Data type mapping. Data types are one of the main issues when it comes to interoperability between different programming languages. There must be an agreed mapping between the data types in the programming language X and the data types in the programming language Y. One way to solve this problem is a mapping table with the different data types supported by the different programming languages.

Object representation. There must be a standard way of representing objects, either the object is written in Java, C#, or other object-oriented programming language. A standard format must be established to represent the supported structures in the different programming languages: classes, primitive data types, arrays, and user-defined classes. This must also include a standard way to represent remote object references.

Messages. They represent the way clients and servers communicate. Messages are used to make requests or receive responses, and they must also be written in a standard way to be understood by clients and servers.

Serialization and de-serialization. In the context of data storage and transmission, serialization is the process of rendering an object into a state that can be saved persistently into a storage medium, such as a file, database, or a stream to be transmitted through the network. De-serialization is the opposite process, which puts the serialized version of the object into a live object. Serialization and de-serialization are processes heavily used when dealing with distributed object-based systems.

In this paper we present Web Objects in XML (WOX), an efficient and easy to use XML serializer for interoperability between the C# and Java programming languages. The WOX serializer is a stand-alone library based on XML (woxSerializer.jar for the Java programming language, and woxSerializer.dll for the C# programming language), which is able to serialize Java and C# objects to XML and back again. One of its main features is the generation of standard XML, which is language independent. This means that if we serialize a Java object to XML, we could take the generated XML to reconstruct the object back to C#; and viceversa. It is worth noticing that our serialization approach does not need code annotations or any other type of configuration from the user in order to serialize objects.

We have made our WOX serializer publicly available as an open-source project, which can be downloaded from http://woxserializer.sourceforge.net/. It must be noticed that the WOX serializer presented in this paper is an improved and interoperable version of the serializer that is part of the WOX framework reported in [Jaimez 2007], which has been used to develop distributed object-based applications. Many people have been using our WOX interoperable serializer since we made it available. As a consequence of this work, the MSDN Magazine (a Microsoft Magazine) published an article, in its October 2010 issue [Khan 2010], about interoperability between Java and .NET, where they mention and recommend the use of our work for interoperability between applications. We consider this as a valuable example that demonstrates that our work has had an impact on the MSDN community.

The rest of the paper is organized as follows. Existing XML serializers are discussed in the next section. A following section describes the main features and uses of the WOX serializer. Then, we dedicate another section to the implementation of the WOX serializer. The XML representation generated by the WOX serializer is presented in a subsequent section along with some examples. Finally, conclusions and further work are discussed in the final section.

Existing XML Serializers

We have not found any out of the box XML serializer to interoperate between different programming languages. In this section we present a series of frameworks and libraries to serialize objects to XML. All of the libraries explored (except one of them) are not able to generate a standard XML object representation to interoperate between different programming languages.

XStream. This is a Java library [XStream 2010] to serialize objects to XML and back again. XStream is able to serialize most objects without the need for custom mappings. The XML generated is easy to understand. One of its features is the use of aliases, which offer a way to use different tags or attribute names in the XML produced. Class types, package names, and field names can be mapped to XML tags. This library is a good option to have an object serialized to XML, but the main limitation is that it is only implemented in Java.

Koala. Koala XML serialization [Koala 1998] is a Java application that provides a way to serialize and de-serialize any Java objects in an XML document. The serialization mechanism of Koala relies in the Java Serializable interface. This application is called KOML for Koala Object Markup Language, which is also a 100% pure Java solution. The website of this project has not been updated since 1998.

XMOP. XML Metadata Object Persistence [XMOP 2000] allows some degree of interoperability between object technologies such as Java, and Microsoft COM. In the case of Java, it provides automatic serialization capabilities only for simple classes. For those classes that contain string and array members, developers have to hand code the serialization methods. In the case of COM, it does not provide automatic serialization for simple nor complex objects. Developers have to code COM serialization/persistence through some interfaces provided. XMOP employs the Simple Object Definition Language (SODL) and its DTD. Using introspection an object is serialized in the XMOP document format, which is a description of the interfaces, properties and methods of an object. Although the XML representation generated by XMOP is interoperable between programming languages, the developer has to do considerable effort to hand code the serialization methods for the objects to be serialized.

JSX. Java Serialization to XML [JSX 2002] serializes only Java objects to XML. JSX can persist any Java object in a human readable format, where the XML generated is nice and clean. Another of its benefits is that it provides the evolution techniques of the Java Object Serialization. This library used to be free, but it now provides a trial period of 30 days, after which the user must pay to use it.

Castor. The Castor project [Castor 2010] is an open source data binding framework for Java. It provides Java-to-XML binding and Java-to-SQL persistence. The Java-to-XML framework enables the user to deal with the data defined in an XML document through an object model which represents that data. Castor can marshal almost any Java object to and from XML. One restriction is that the Java classes of the objects to be serialized must follow the Java Bean conventions. Castor uses a set of class descriptors and field descriptors to describe how an object should be serialized and de-serialized from XML. Castor can work in two different modes: the introspection mode does not require any configuration from the user; but in the mapping mode, the user provides a user-defined mapping file that allows the definition of a customized mapping between Java classes and XML.

JAXB. The Java Architecture for XML Binding [JAXB 2010] is a framework for processing XML documents. Unmarshalling an XML document with JAXB results in a tree of objects, with the nodes in this tree corresponding to XML elements, which contain attributes and the content as instance variables. In order to extract this information and convert it to objects from the XML document it is needed an XML Schema (which defines the structural relationships and data types). The XML Schema is compiled, and a set of Java classes is generated, which define the types required for accessing elements, attributes and other content. In this approach the user is required to know the structure of the XML document, and create an XML Schema in order to map it to Java objects.

Although there are other existing libraries to serialize objects or messages to XML and viceversa, they are not stand-alone, but they are part of a framework for web services, such as XML-RPC [Winer 1999], or those that implement the SOAP standard [SOAP 2003]. Web service frameworks like those use XML serialization internally, and are out of the scope of this paper.

The next section presents some of the main features of our WOX serializer, which can be used to generate standard XML representations of objects, and is able to interoperate between C# and Java.

WOX Features and Uses

The main features of the WOX serializer are listed below.

The WOX serializer can be used by any program that needs to interoperate between C# or Java, or simply to serialize objects to XML from any of those programming languages. Some of the purposes where the WOX serializer can be used are listed below.

WOX Implementation

The implementation of the WOX serializer is described in this section through three different flow charts, which illustrate how the WOX serializer writes any object to XML. This section has also been presented in [Jaimez 2011]. The flow chart in Figure 1 shows the base functionality. There are some notes indicated with numbers inside a dotted square (A1 to A7) which are explained below.

Figure 1: Flow chart A

Flow chart A: Base of the WOX serializer.

A1. We represent a NULL object as an EMPTY object in XML.

A2. We use a Map to store object references. This avoids the duplication of objects in the XML representation, and the unnecessary serialization processing of objects that were already serialized. This Map is also used to handle adequately recursive object references, because the serializer puts an object in the Map as soon as it goes through it (it is the first step in the serialization process). This means that if this object is nested inside another object, the serializer will not serialize it again, because when it reaches this object, the serializer will find out that its reference is already in the Map, which handles the problem of recursive references.

A3. If the object is already in the Map, it is not necessary to serialize it again (a reference to the object will be stored instead). The Map is populated as the serializer finds new objects. The idref XML attribute specifies the object id in the Map that corresponds to the object found. It is important to notice that the id and idref attributes are not the same as those normally used in the XML sense; but they are used to identify uniquely objects that have already been serialized.

A4. We say that an object is stringable if it is of any of the following classes: Byte, Short, Integer, Long, Float, Double, Character, Boolean, Class, String.

A5. Other objects in their stringable versions can be seen below.

<object type="boolean" value="true" id="11" />
<object type="char" value="\u0009" id="15" />
<object type="int" value"785" id="21" />

A6. The serializer will process the array. The process to serialize an array is illustrated in the Flow chart B shown in Figure 2.

A7. At this point the serializer has found another type of object (not a stringable object, not an array). It is a user-defined class or other type of object not covered in the previous cases. The serializer will write this object, and go through each of its fields (attributes) to serialize them. The process to serialize each of the object fields (attributes) is illustrated in Figure 3.

The flow chart in Figure 2 shows the functionality of the WOX serializer to deal with arrays. The flow chart has some notes indicated with numbers inside a dotted square (B1 to B3), which are explained below.

Figure 2: Flow chart B

Flow chart B: Arrays.

B1. The serializer checks the class of the object against an array that holds all the possible primitive uni-dimensional arrays.

B2. It is not a uni-dimensional array. It could be a n-dimensional array, or an array of objects of user-defined classes.

B3. The type of the primitive array found is any of the following: int, boolean, short, long, char, float or double. Some examples of primitive arrays serialized to XML by the WOX serializer are shown below.

<array type="int" length="6" id="10">45 67 78 32 34 79</array>
<array type="char" length="3" id="19">\u0065 \u0004 \u0067</array>

The flow chart in Figure 3 shows the functionality of the WOX serializer to deal with fields of an object. The flow chart has some notes indicated with numbers inside a dotted square (C1 to C3), which are explained below.

Figure 3: Flow chart C

Flow chart C: Fields of an object.

C1. The WOX serializer considers the following as primitive data types: byte, short, int, long, float, double, char, boolean.

C2. In this case stringable objects are treated as fields of a parent object. A stringable object is not labeled as object when it is the field of another object (observe that it also lacks of the id attribute). This aims to have a more compact encoding.

C3. An example of the XML generated for a field of an object is shown below. The field is an object of type ser.Person, which also has two more fields: name of type string, and age of type int.

<field name="fieldName">
   <object type="ser.Person" id="9">
      <field name="name" type="string" value="Carlos" />
      <field name="age" type="int" value="30" />
   </object>
</field>

WOX XML Representation

In this section we will introduce the data type mapping table used by WOX, and show the XML representation of different types of WOX objects: simple objects with primitive types, uni-dimensional primitive arrays, multi-dimensional primitive arrays, object arrays, lists, maps, and user-defined objects (which also have fields with different types of objects).

Simple object with primitive types

In this subsection we illustrate the XML representation of WOX primitive types, which include: byte, short, int, long, float, double, char, and boolean. The complete list of the data types supported by WOX is shown in Table 1.

Figure 4 shows the Product class, which has five fields of primitive types: name of type string, price of type double, grams of type int, reg of type boolean, and categ of type char.

Figure 4: Product class

Product class with primitive fields.

The XML representation in WOX of a Product object is shown below. The XML is simple and clean.

<object type="Product" id="0">
   <field name="name" type="string" value="Corn" />
   <field name="price" type="double" value="3.98" />
   <field name="grams" type="int" value="500" />
   <field name="reg" type="boolean" value="true" />
   <field name="categ" type="char" value="\u0041" />
</object>

The root of the XML document is an object element, with its type attribute equals Product, which is the class of the object. The id attribute is used to handle object references (in this case we only have one object - the product object). Every field in the product object is represented by field elements, which have the following attributes: name (the name of the field in the class), type (the WOX data type of the field), and value (the value of the field for that particular object). Primitive types are represented in WOX as field elements.

Observe that the categ field is of type char. WOX data type char is represented as a Unicode value.

Uni-dimensional primitive arrays

Uni-dimensional primitive arrays contain primitives as their elements. Primitives include the following data types: byte, short, int, long, float, double, char, and boolean. For a complete list of the data types supported by WOX see the Data types mapping table.

Primitive arrays are treated differently from other arrays, because they are serialized in a very efficient way. We will show two examples of the XML representation for primitive arrays. The first one is a stand-alone array, in which the array itself is the root object. In the second example, the primitive array is part of an object (it is declared as a field in a class).

The XML representation of a stand-alone primitive array is shown below. The XML is simple and efficient. The root is an object element, with its type attribute equals array, which indicates that the object represented in this case is an array. The elementType attribute provides the data type of the primitive array, and the length attribute indicates the number of elements in the array. The id attribute is used to handle object references (in this example we only have one object - the array itself). The elements of the array are separated by spaces, and presented as only one string.

<object type="array" elementType="double" length="5" id="0">
   12.45 878.98 987.98 435.87 537.87
</object>

The XML representation of an object that contains three uni-dimensional primitive arrays as fields is shown below. The primitive arrays are declared as fields of the TestArray class, which is illustrated in Figure 5.

Figure 5: TestArray class

TestArray class with three primitive array.

In this case, the arrays are not root objects, but they are now fields of the root object. The codes field is a primitive array of char elements, the values field is a primitive array of int elements, and the answers field is a primitive array of boolean elements. It should be noticed that arrays are also objects with their type attribute equals array.

<object type="TestArray" id="0">
   <field name="codes">
      <object type="array" elementType="char" length="5" id="1">
         \u0065 \u0074 \u0072 \u0067 \u0077
      </object>
   </field>
   <field name="values">
      <object type="array" elementType="int" length="5" id="2">
         23 56 78 33 69
      </object>
   </field>
   <field name="answers">
      <object type="array" elementType="bool" length="5" id="3">
         true false true false false
      </object>
   </field>
</object>

The root is an object element, with the type attribute equals TestArray, which is the class of the root object. The root element has three field elements as children, which correspond to the three primitive arrays. Each array is a field of the root object, and they are treated as object elements, with type, elementType, lenght, and id attributes. The elements of each array are separated by spaces and written as one simple string.

Multi-dimensional primitive arrays

Multi-dimensional primitive arrays can also be serialized as stand-alone arrays, or as part of an object. Below we show the XML representation of an object of the TestMultiArray class. The TestMultiArray class, which is illustrated in Figure 6, has one bi-dimensional array of int elements as its field.

<object type="TestMultiArray" id="0">
   <field name="matrix">
      <object type="array" elementType="int[]" length="3" id="1">
         <object type="array" elementType="int" length="5" id="2">23 56 89 36 68</object>
         <object type="array" elementType="int" length="4" id="3">87 64 88 32</object>
         <object type="array" elementType="int" length="6" id="4">78 80 21 29 34 67</object>
      </object>
   </field>
</object>

Figure 6: TestMultiArray class

TestMultiArray class with one multi-dimensional primitive array as field.

We can observe that the root is an object element, with its type attribute equals TestMultiArray, which is the class of the root object. The root element has only one child: a field element, which is a bi-dimensional primitive array. Since this is a bi-dimensional array, it is actually serialized as an array of arrays. It is an array of three elements of type int[], where every element is an array of type int. The first array has five elements, the second array has four elements, and the last array has six elements. The elements of each array are separated by spaces and presented as one simple string.

It should be noted that every object element has an id attribute, which is used to handle object references. Arrays of more than two dimensions (multi-dimensional) are represented in XML following the same idea shown in this subsection.

<object type="array" elementType="Product" length="3" id="0">
   <object type="Product" id="1">
      <field name="name" type="string" value="Beans" />
      <field name="price" type="double" value="1.75" />
      <field name="grams" type="int" value="250" />
      <field name="reg" type="boolean" value="true" />
      <field name="categ" type="char" value="\u0042" />
   </object>
   <object type="Product" id="2">
      <field name="name" type="string" value="Rice" />
      <field name="price" type="double" value="3.89" />
      <field name="grams" type="int" value="750" />
      <field name="reg" type="boolean" value="true" />
      <field name="categ" type="char" value="\u0052" />
   </object>
   <object type="Product" id="3">
      <field name="name" type="string" value="Bread" />
      <field name="price" type="double" value="1.06" />
      <field name="grams" type="int" value="300" />
      <field name="reg" type="boolean" value="false" />
      <field name="categ" type="char" value="\u0048" />
   </object>
</object>

Lists

Lists are similar to arrays in WOX. A List in WOX is the equivalent to java.util.ArrayList in Java, and System.Collections.ArrayList in C#, as can be seen in Table 1 previously shown. A List in WOX is represented in a simple form, and can be de-serialized either to Java or C#.

The XML representation of a list of Course objects is shown below. The Course class, which is illustrated in Figure 7, has three fields: code of type int, name of type string, and term of type int.

<object type="list" elementType="Object" length="3" id="0">
   <object type="Course" id="1">
      <field name="code" type="int" value="6756" />
      <field name="name" type="string" value="XML" />
      <field name="term" type="int" value="3" />
   </object>
   <object type="Course" id="2">
      <field name="code" type="int" value="9865" />
      <field name="name" type="string" value="DB" />
      <field name="term" type="int" value="2" />
   </object>
   <object type="Course" id="3">
      <field name="code" type="int" value="1134" />
      <field name="name" type="string" value="Java" />
      <field name="term" type="int" value="2" />
   </object>
</object>

Figure 7: Course class

Course class with three fields.

The root is an object element, which represents the list of three Course objects. Each course has its type, and id attributes, and its three field elements (children): code, name, and term. Every field element specifies its name, type, and value attributes.

<object type="map" id="0">
   <object type="entry">
      <object type="int" value="1134" id="1" />
      <object type="Course" id="2">
         <field name="code" type="int" value="1134" />
         <field name="name" type="string" value="Java"/>
         <field name="term" type="int" value="2" />
      </object>
   </object>
   <object type="entry">
      <object type="int" value="6756" id="3" />
      <object type="Course" id="4">
         <field name="code" type="int" value="6756" />
         <field name="name" type="string" value="XML" />
         <field name="term" type="int" value="3" />
      </object>
   </object>
   <object type="entry">
      <object type="int" value="9865" id="5" />
      <object type="Course" id="8">
         <field name="code" type="int" value="9865" />
         <field name="name" type="string" value="DB" />
         <field name="term" type="int" value="2" />
      </object>
   </object>
</object>

Object references

In order to avoid duplicate objects in the XML representation of an object, the WOX serializer uses object references, which are handled by using id and idref attributes. Figure 8 shows an array of Product objects with duplicates.

Figure 8: Object references

Array of Product objects with duplicates.

The use of object references in the XML representation of this array of Product objects is illustrated below. The Product class was introduced previously, which has five attributes: name of type string, price of type double, grams of type int, reg of type boolean, and categ of type char.

<object type="array" elementType="Product" length="6" id="0">
   <object type="Product" id="1">
      <field name="name" type="string" value="Beans" />
      <field name="price" type="double" value="1.75" />
      <field name="grams" type="int" value="250" />
      <field name="reg" type="bool" value="true" />
      <field name="categ" type="char" value="\u0042" />
   </object>
   <object type="Product" id="2">
      <field name="name" type="string" value="Rice" />
      <field name="price" type="double" value="3.89" />
      <field name="grams" type="int" value="750" />
      <field name="reg" type="bool" value="true" />
      <field name="categ" type="char" value="\u0052" />
   </object>
   <object idref="1" />
   <object type="Product" id="3">
      <field name="name" type="string" value="Bread" />
      <field name="price" type="double" value="1.06" />
      <field name="grams" type="int" value="300" />
      <field name="reg" type="bool" value="false" />
      <field name="categ" type="char" value="\u0048" />
   </object>
   <object idref="3" />
   <object idref="1" />
</object>

The duplicated Product objects in the array are not duplicated in the XML representation that WOX generates. They are referenced by using the idref attribute, which actually refers to the unique id given to every object. It can be observed that the XML representation of the array has the three object references for the duplicated objects (two for product p1 identified by id="1", and one for product p3, identified by id="3".

Conclusions and Further Work

In this paper we have presented Web Objects in XML (WOX), an efficient and easy to use XML serializer for interoperability between the C# and Java programming languages. The WOX serializer is a stand-alone library based on XML which is able to serialize Java and C# objects to XML and back again. One of its main features is the generation of standard XML, which is language independent. This means that if we serialize a Java object to XML, we could take the generated XML to reconstruct the object back to C#; and viceversa. Our serialization approach does not need code annotations or any other type of configuration from the user in order to serialize objects.

We have made our WOX serializer publicly available as an open-source project, which can be downloaded from http://woxserializer.sourceforge.net/. As a consequence of this work, the MSDN Magazine (a Microsoft Magazine) published an article, in its October 2010 issue [Khan 2010], about interoperability between Java and .NET, where they mention and recommend the use of our work for interoperability between applications. We consider this as a valuable example that demonstrates that our work has had an impact on the MSDN community.

In this paper we described the main features and uses of the WOX serializer, and we also dedicated a section to give its implementation details. We showed the XML representation for the most representative structures and objects in both Java and C#, along with some examples. We believe that the XML object representation that WOX uses is efficient, clean, easy to understand, and can represent objects in other object-oriented programming languages. Further work is needed to develop our XML serializer in other object-oriented programming languages, and allow interoperability among them. We have already started our analysis to develop a WOX serializer for the C++ object-oriented programming language.