-->Java Native I/O

Existing Approaches

The general task of all frameworks and tools reviewed here, is to provide access to a C struct or sequences of such, given in a byte stream. Main property of this structured data type is, that all member variables are physically grouped in one continuous block of memory. Even arrays or nested structs are in the same block not references to other memory areas.

The order of data inside structs is defined by the order of their declaration. If one member variable precedes another in the declaration of the struct, it will do so in memory as well.

The position of a member relative to the start of the data block, depends on the order of members in the structure and their alignment/padding. In C, there is a standard alignment relative to physical memory addresses for basic types and structs, defined by the systems application binary interface (see e.g. Unix System V Application Binary Interface), which is mainly motivated by performance concerns in respect to the systems' hardware. The developer can explicitly declare different alignments for every type or variable he/she introduces. Thus, each member of the struct can have a specific alignment relative to physical memory addresses and this is the usual case, when data is written to byte streams, to remove unnecessary padding.

In order to provide access to the data in a type-safe manner, the frameworks, tools and concepts reviewed here have to have the following building blocks:

1. Anything to either define or create and/or use a Java interface, which resembles the C struct.
2. An encoding and decoding subsystem to handle access to variables in the struct in respect to its position and alignment which translates into an offset.

The challenge hereby, is the lack of a Java type, which resembles the same properties as a structured type in C. Mainly because nested types and arrays will always be references. Therefore, the most practical approach, is to exchange the whole structure by value, only. However, the major disadvantage of this approach is the wasted processing time when only a minor subset of the data is actually touched by the application.

The review focuses on the following properties of applied approaches:

• Type declaration: How much effort goes into describing/declaring the struct and a corresponding Java interface and is it flexible enough to cover all use cases. Also, how much effort will go into learning.
• User error prevention: Is the provided Java interface robust against possible mistakes or misunderstanding by the user or is it considered error prone in this respect? For example, providing access to an array which is a view on the buffer via a reference can be mistaken by a user, who thinks this array does behave like a copy and keeps the reference. This consequently leads either to memory leaks or data corruption, depending on the implementation of the link between buffer and view.
• Impact on call site: Does the tool have to alter code blocks in the application which access struct types? This is more a question of trust -- trust in the code which is injected in the application.
• Impact on overall performance: How much additional memory is required and what effect will it have on cache performance in terms of cache misses? What is the expected performance impact, in respect to initialisation of subsystems at application start and encoding, decoding at runtime?

Those figures will be rough estimations, only. An actual evaluation is not part of this review.

LWJGL v3: Struct

LWJGL uses a class called Struct to access data in byte streams. Derived classes are created by an offline code generator at development time. Offline generation requires an additional step during build, but there is no instrumentation of the call site.

Classes for mapped objects are derived from class Struct and provide methods, which do offset handling and data access for member variables of the struct. Access to member variables is performed via methods only. All data is returned by value, which circumvents direct access to references such as arrays.

There is a helper class called Layout to declare offsets of member variables during initialisation of a particular struct class. This layout information is used only to calculate offsets and sizes of members identified by indices. Layout information is discarded after initialisation and not designed to keep application specific meta-data.

Offsets or member variables are stored in static final members of the struct class, which are treated as constants by the JIT compiler and as such inserted in the code (text cache not data cache). An instance of the struct class is then attached to a memory location which might be either a byte buffer, or memory on stack or in a native library. But as long as the mapped object stays in memory, the buffer is not garbage collected.

To access a sequence of structs there is a similar class StructBuffer which provides the same functionality in terms of access to members of structs and additional functionality to iterate over the sequence or skip to a particular instance in the sequence.

LWJGL v2: MappedObject (legacy)

In LWJGL v2 exists a class MappedObject which allows to define struct types in Java which can be mapped to byte buffers. It very much looks like this is derived work from the author of LibStruct (see below) but was discarded in LWJGL 3, to focus more on LWJGL's core functionality, which is binding.

Mapped objects are declard as Java classes derived from MappedObject. Classes can use a @MappedType annotation, to explicitly declare alignment/padding of instances of the class and its fields. Mapped object classes have to be registered, once.

The declared mapped object can have even public member variables and integrates seamlessly with the remaining code. This comes at the cost of more responsibility on user side. The user needs to understand, that this object (especially references on members such as arrays) can only exist as long as the backing memory is available, otherwise he will run into errors, which he can't locate and fix without the missing knowledge.

The mechanism relies on runtime code generation/instrumentation which is triggered through a special class loader MappedObjectClassLoader, which must be present in the current thread. The class loader delegates to code generation when loading affected classes. Code generation/instrumentation affects classes derived from MappedObject and classes which use those classes to allow direct access to references such as arrays (i.e. call site has to be instrumented).

Offsets are inserted as constants in the generated code, which puts them in text cache not data cache.

LibStruct

The LibStruct project follows a very similar approach to MappedObjects in LWJGL 2.

Classes of mapped objects (here structs) are declared by adding the annotation @StructType. Other annotations are used to declare semantics of its members:

• @StructField: Used to declare a field to be part of the struct. Fields without it will be considered to be transient in respect to the structs layout.
• @CopyStruct: Used to declare methods to return a copy of the struct.
• @TakeStruct: Used to declare methods which return a reference instead.

Methods can have a body which operates on member variables of the struct before returning a value. Declared struct types can have public member variables (even references).

Again, struct types integrate seamlessly with the surrounding code, which leads to the same risks on user side as described for MappedObjects when accessing references to arrays or nested structs.

LibStruct supports runtime code generation and instrumentation through a Java agent, but comes also with a tool StructPacker, which performs offline code generation and instrumentation for a release version of the application. This is in fact a final optimisation step to be performed on the whole application, which removes all of the code generation and instrumentation effort entirely.

The concept requires generation of code to handle offsets in data access for each struct class and instrumentation of all classes using the mapped struct objects to allow direct access to members. When using runtime code generation, it also requires a class loader, which triggers instrumentation of classes using structs.

Java Binary Block Parser

The Java Binary Block Parser (JBBP) aims to provide a framework which supports access to structs with variables of any integer type (even unsigned) but no floating point types, for example. Strength of the framework is its support of mapping single bits to variables.

JBBP provides a domain specific language to declare those struct types or even new simple types such as three byte integers (int24). The type declaration is provided to the API partly in annotations and partly in strings which makes the whole process prone to errors and those errors are hard to find and solve. JBBP does no code generation and thus everything has to be coded by the user using an API consisting of a lot of methods for special cases.

Since version 1.3 (Sep. 2017) JBBP supports offline code generation, which improves performance of data access at runtime. The declaration of structs did not change.

Javolution: Struct

Javolution is a framework aiming to provide tools for the development of realtime and embedded systems. Support of access to unstructured byte streams is provided through base classes Struct and Union which do offset handling but there is no code generation.

Structs are declared as classes derived from Struct. A struct class effectively encapsulates access to the data in the byte stream and there are no public member variables. In fact, everything is encapsulated. Even each primitive type has a corresponding class to encapsulate it. This has the advantage, that additional types, such as Unsigned16, can be integrated seamlessly with the API and it potentially provides control over references.

It introduces classes for all standard primitives types in C and similar classes can be declared for nested structs or arrays with appropriate getters and setters for their value(s). This strategy establishes a concept which is consistent in its structure and semantic. When accessing a reference on a nested struct, the user gets a view object on this struct which cannot be mistaken as a copy, for example.

The lack of a code generator results in much development effort for struct types, almost the same effort as writing the functionality without the helper classes. The approach to have an object instance for every single attribute in the struct causes a lot of memory management overhead, increased footprint and potentially increase in cache misses, because of the wide spread distribution of data in heap memory.

Preon

Preon calls the problem by its name and aims to provide a codec infrastructure for struct types with their library. Mapping of byte streams to Java types is in fact bound to the tasks of encoding and decoding of data. Preon does not provide code generation or instrumentation but a strong concept and classes to implement such tasks.

Struct types are declared as simple Java classes and each member variable to be considered in mapping, has to be explicitly marked with an annotation @Bound. This applies to all types whether it is a primitive type, array or a compound type (nested struct). Preon also considers inherited attributes of base classes of the struct. Unfortunately, they have forgotten to consider union types. However, what makes Preon special is its support of polymorph types: For example, the user can declare attributes to be part of the struct only if a given expression over previous attributes in the struct evaluates to true.

At runtime, the application requests a codec from the framework, providing its annotated Java type as reference. The codec creates a new instance of the given Java type and decodes the data into it at once. Where possible it does lazy decoding, which is achieved through proxies (for example for nested structs), which are inserted in the object representing the struct in Java. But it usually copies a lot of data at once.

The processing effort to instantiate a codec for a given type is presumably pretty high because each time a codec is requested, the given struct class is searched for annotations and appropriate codecs for attributes are instantiated. And having object instances for each attribute results in the same kind of distribution of data in heap memory as in Javolution and therefore the same impact on cache misses etc.. But the development effort is lower compared to Javolution.

Simplified Wrapper and Interface Generator (SWIG)

A different approach to perform access to native data is using JNI. Because JNI supports only generation of C header files from Java classes, which lacks the ability to define alignment etc., the most flexible approach is to use a framework such as SWIG, GlueGen or JNA (see below).

In SWIG, struct types can be declared in C with support of all the features of C structs. A code generator then generates the corresponding counterpart in Java and C code to access the data in JNI code (see documentation). Thus, the application will need to load this native library at runtime and provide the pointer to the byte buffer through JNI to get an instance of the struct object. This object then has native getter and setter methods, and each call to those methods goes into the library, where the actual access to data is performed and then translated to Java byte ordering on the way back through the JNI layer.

Native calls come at a certain cost, which is not just related to byte order manipulations. But this cost is presumably very close to pure Java approaches, if properly optimised (TODO: analyse), but the main disadvantage for many users will be elsewhere:

• The setup of the development environment with code generation step and additional compiler.
• Writing the type declaration in C with the special extensions of SWIG, which have to be learned first.
• Having an additional library which has to be (cross) compiled for different platforms and linked accordingly, at runtime.

JogAmp: GlueGen

GlueGen is part of the JogAmp project. It is a code generator which generates Java and JNI/C code to call C libraries.

GlueGen requires C header files with declared C functions and data types and corresponding configuration files as input to generate (they call it 'emit') appropriate Java native methods/classes and JNI interface code. The configuration file provides vital information, such as C functions and data types to be considered and the target package and class names for generated code and fine grained control over more detailed options such as ignoring fields of structs or the scope of visibility of generated native methods (public/private etc.). Conversion of data between Java and C occurs in the generated JNI code.

Java Native Access (JNA)

JNA is another framework, which provides interfacing to native functions or data, but unlike SWIG and GlueGen the API user writes all of his/her code in Java. In terms of struct types, the framework provides classes to declare and access native data in for example byte buffers.

Declared struct types are classes derived from Structured (see javadoc). Instances of those classes act as copies of the data behind it. Data has to be explicitly copied forth and back through methods of the base class Structured such as read, write. If the application needs just a subset of the data, then single fields of the struct object can be updated through the method readField(String). Thus, there will be a string comparison on each access to that field and it heavily relies on reflections at runtime to set the corresponding attribute in the struct object. Also, since read data is always stored in a field of the corresponding Java object, there will be a detour via the heap, even if the data is actually just needed in the current context (stack/registers).

Project Panama

Project Panama, officially announced in June, 2014 by John Rose (Oracle), aims to improve interconnections between the JVM and native libraries in general. Current prototype of this project can be found under the name Java Native Runtime. In respect to struct types, they follow a very similar approach to Javolution, by the time of this review: Every single attribute of a struct is boxed, which has the disadvantages already mentioned in regards to Javolution. In his blog, the project lead states, that he expects "that value types will narrow the gap eventually for other C types, but they are not here yet" (refer to next section for value types). So, there might be a chance that the approach on access to struct types will change.

Java Value Types

Last but not least, there are plans to support struct-like types in a future version of the Java language. First concepts refer to such new data types as Value Types which have almost the same properties as structs in C. There is also an approach to flatten arrays, such that a multi-dimensional array doesn't have to be physically an array of references on other arrays. If those two approaches make their way into the Java language specification, there will be support to represent the data in a more compact form, but that does not involve interfacing or mapping to byte streams. Also, by now, there are no signs of support to control alignment of data or having unions or mapped bits. So, whether it will be useful for such purposes, is unknown for now.