Combine app-defined payloads with lightweight DIME format

Direct Internet Message Encapsulation (DIME) offers a better alternative to MIME and raw XML to combine multiple text and binary records into one binary stream. See how DIME can work with your applications.

The Direct Internet Message Encapsulation (DIME) format provides an easy, standard mechanism for combining multiple text or binary data records into a single binary stream. Records are stored in an encoding-independent fashion designed for efficient parsing and easy implementation. You can find the DIME specification here.

DIME is better than MIME and raw XML
Of course, other systems provide for the creation of a single message incorporating multiple text and binary portions. But each was optimized for a specific problem space, which makes them less than ideal for non-e-mail, intra-application communication.

The most obvious alternate is Multipurpose Internet Mail Extensions (MIME) as defined in RFCs 2045 through 2049. MIME is the e-mail standard for file attachments, which are just messages with text and binary portions. The unsuitability in the general case stems from MIME's e-mail origins. E-mail was designed to travel through systems that handle only text. Consequently, MIME works by converting both text and binary portions into an internally delimited all-text portion. This conversion to text introduces cross-platform character set and encoding issues, as well as increasing the size of binary components by a significant fraction.

Raw XML provides another mechanism for representing binary information among text—the CDATA type. CDATA blocks are not parsed by XML parsers and therefore don't meet the "well-formedness" requirements of an XML document. Converting binary data into base 64 or similar encodings and including it in CDATA sections allows you to send poorly formed XML.

A DIME buys you a bit of code
Unlike MIME and CDATA sections in XML documents, DIME converts both text and binary portions into a single binary message. This storing of the binary portion in its raw binary form saves the DIME parser the overhead and complexity of reversing a text to binary encoding, as is required with MIME and CDATA/XML. Indeed, the simple length-delimited nature of the DIME parts, called records, allows the DIME parser to split the records without looking inside, past an initial binary header.

Listing A shows some simple code for creating a DIME message using a freely available Java DIME implementation. The first few lines create a new DimeGenerator object that is constructed with a FileOutputStream passed into it as its output source. The second and third blocks of code each add a DIME record created from an on-disk text file. Notice the use of the MEDIA_TYPE TypeNameFormat and the "text/plain" and "image/gif" MIME types used to indicate the formats of the DIME Records' contents. MIME type is but one of the ways to provide information about the data contained within a DIME Record outlined in the DIME specification, but it's perhaps the most useful.

The last block of code in Listing A ends the DIME message and closes the file to which it was written. The DimeGenerator class uses streams throughout for all I/O operation, which eliminates any need for large amounts of memory to store DIME records during message construction. Also, notice the final Boolean parameter to both of the addRecord method calls in the example. A true value indicates that the record being added to the DimeGenerator is the last record. Adding subsequent records results in an IllegalStateException.

Listing B shows code that can be used to parse the .dime file created by Listing A. You can see that the DimeParser class provides a java.util.Iterator-like interface that can be used to extract DimeRecord objects. In addition to providing methods that expose the MIME type metadata stored within the DIME record, the getPayload method makes available an InputStream that produces the data stored within the DIME records. As is the case with the DimeGenerator, the payload data is never stored in memory; instead, streams are used throughout, allowing for large payloads with low memory consumption.

Why DIME might be a good fit for many applications
DIME is an excellent fit for applications that need to send binary and text data through the same stream. This is often required when XML messages have referenced attachments. Microsoft has authored a specification for using DIME and SOAP messages to replace the SOAP with attachment standards. When using SOAP and DIME, you can provide textual information about the method invocation and its result using standard SOAP XML, while using a DIME binary record for any nontext information contained in the calling parameters or return result. Since both the SOAP and DIME standards are now available as Java libraries, you can interact with Windows SOAP+DIME Web services in their own language, from any platform.

Editor's Picks