As developers, we’re always striving to produce modular, easily extended designs. Unfortunately, we often don’t know whether we’ve succeeded until the implementation is complete and we’re asked to change something. In this article, I’ll describe the life of a small application from a poor, inflexible design through its current incarnation, which takes advantage of open standards.

First step: Client request and prototype
As part of a larger project for tracking calls to our Customer Service Centers, our clients requested a tool for filling out form letters to be sent to customers.

The project began with a prototype of an application to fill in the blanks on a form letter. The user interface was a split window, with a preview of the completed letter in the top half and entry fields in the lower half. Significantly, this interface has remained virtually unchanged throughout the life of the application.

When the clients initially gave us the letters they wanted to use, they merely used asterisks to enclose the text users would complete. The prototype continued this convention by creating a simple parser that looked for asterisks in the input text and created text fields for them. So, for example, the input file in Listing A would appear as shown in Figure A.

Figure A
Example input

This was fine, but once the clients saw the prototype, the requests came rolling in:

  • ·        “We’d like you to validate things like telephone numbers, dates, and ZIP codes.”
  • ·        “We’d like to provide default text.”
  • ·        “We’d like to limit the length of some fields.”
  • ·        “We’d like to have repeatable fields arranged in a table.”
  • ·        “We’d like to have either/or choices.”

For each of these situations, we used more punctuation marks to indicate the characteristics of each field. At that point, the same letter template looked like the one in Listing B.

This worked well for a few years, but eventually it became obvious that we needed a better way to describe the letters. If nothing else, we would soon run out of punctuation marks. Clearly, we were suffering from the lack of a forward-looking design in the initial phases of development.

Time for a rewrite
When we finally had a list of needed enhancements that couldn’t work with the current design, I decided it was time to rewrite the parser. To do this, I used the well-proven UNIX tools, Lex and YACC. I devised a language for marking up the letter text, based loosely on the HTML Form tags. I added attributes to provide for our needs. Our example letter template now looked like the one in Listing C.

One of the major steps forward was that we could now assign internal names to each field. In the past, all fields were anonymous; they were just text. Now they had identities. This allowed us to integrate a number of new features, like prefilling the customer’s name and address fields with data from the main application. Or, we could print the address on an envelope at the same time that we printed the letter, or fill in a fax cover page.

By using HTML-like tags, we added some much-needed structure to the language. From that point on, it could evolve naturally. New features only required new attributes or tags. Using Lex and YACC meant that we could make changes to the language easily by just adding new rules that fit into the existing syntax.

Another advantage of the new design was in the use of tables. A table is a row of fields that can be repeated as many times as needed for a particular customer. Initially, the tables were indicated like this:
*#[dir_code]{20}DIRECTORY|[issue]{5}ISSUE|[money]{11}MONTHLY AMOUNT|*

The # introduced a table, and the | separated columns. This created a set of fields for entering a directory name, issue date, and amount. This set of fields could be repeated as many times as the user required, each on a new line in the output. The system simply made sure that the columns lined up.

Now that the table’s fields were simply content for the <table> element, they didn’t have to be lined up in columns. In fact, they didn’t have to be on separate lines. A table was now simply arbitrary text that could be repeated an arbitrary number of times. This was clearly a situation that the previous syntax couldn’t handle.

Small steps in the right direction
Note that it’s not actually true that we converted the format to HTML all at once. While I was working on the design for the new syntax, the client requested the ability to include conditional text. This text would be used only if the client’s customer spoke Spanish, or if it could be based on the customer’s state of residence. The user would be able to select these options from the Editor window, but we needed a way to specify in the letter template which text was optional.

We did it by implementing an <if> </if> construct. For example:
<if span>
Si Usted no entiende o tiene alguna pregunta, llame al 800-555-1234.

This took us a step in the direction we wanted to go, but it still used the old parser. However, it proved that an HTML-like syntax was better suited to any future features. That gave us the kick-start we needed to move forward with the redesign.

Up until this time, all the letter text was displayed and printed using only a Courier font. This made it easy to go from display to printer. However, since the letters went to customers, we wanted something that looked more professional, with more flexible formatting.

The first problem was that we needed two different routines to generate the complete letter text: one to create complex strings for the display and another to produce PostScript code for the printer. (Our application is written for Motif, but the display vs. print problem may be less acute when using Windows.)

We could now use <b>, <i>, or <tt> tags to select different fonts for our letters. The letters could also include hard page breaks where needed.

Once again, as soon as we’d changed the design to allow more possibilities, the possibilities arrived. For example, we now insert control strings in the appropriate places to include the user’s signature, but only if the letter is being faxed, not printed.

Enter XML
I’d been toying with the idea of converting the application to use an XML parser so I could get some experience with XML tools. Then our client asked if we could provide printouts of each letter in the system without having to go into the application and fill in all the required fields for about 120 letters and forms.

Unfortunately, the answer was no. We had long since moved away from a template format that was presentable for previewing. This was the excuse I needed. I knew that if we used XML as the markup language, I could apply an XSLT style sheet and get a nicely formatted preview of the letters.

The good news was that it only took about a week to unlink the Lex/YACC parsing code and write some event handlers for the SAX parser that we chose. I attribute this to the original design, which separated the parsing code from the internal manipulation and the user interface. After another week of learning XSLT, I had implemented two style sheets and a tiny JScript application to allow the users to preview the letters in Internet Explorer, with two different levels of detail.

This rewrite didn’t yield any new features in the letters themselves—it only provided the ability to do more with the letters. However, using a generic XML parser made it even easier to add new features, such as when I later added a new type of input control (a check box).

To get all the current letters into XML format, I wrote a small program linked to our existing parser to read the current template files and output well-formed XML. This also relied on the fact that the parsing code was well modularized in the original application, so it could be reused in the conversion tool.