The SOAP W3C specification rests on a number of other W3C specifications and on terminology typical of communications programmers. The specification language can be an obstacle if you’re just trying to understand SOAP (Simple Object Access Protocol). In this article, I present the core SOAP ideas expressed in a simple Web page and a Perl CGI program. The syntax is familiar (and wrong), but the ideas are right. Learning the ideas first and syntax later is an easy way to go.

What is SOAP?
SOAP, also called XMLP (for XML Protocol), provides a standard way for two programs to exchange information. This is a fundamental necessity if separate organizations are to cooperate electronically.

There are many ways to exchange messages, including e-mail, chat, and Remote Procedure Calls (RPC). E-mail and chat messages aren’t generally computer-friendly. E-mail headers are computer readable, but the typed content is not understood by a silicon brain. The same applies to chat. RPC, on the other hand, is computer readable but not human readable.

Computers do know how to understand XML. SOAP describes how to bundle messages into XML. It also describes who-sends-what-message-where-and-when. That is why it’s called a protocol. SOAP is not separate from e-mail protocols (SMTP), RPC (sockets & IDL), or Web protocols (HTTP). SOAP uses those systems as a starting point for its messages.

Standard form submission under the microscope
The sample files included in Listing A and Listing B (Simple.html and Simple.cgi) are about as basic as anything you’ll ever see.

Listing A contains a form with four fields and a Submit button. Fill in the first three fields, say, with name J. Random, age 17, and hair color brown; press Submit; and the page is redisplayed with the fourth, descriptive field reading, “J. Random is a young brunette”. The background changes to a ‘purty’ yellow, too. Notice the HTML form in Listing A calls the CGI program in Listing B. It performs a remotely implemented service for the browser page. If locally implemented, you’d avoid form submissions and use a heap of JavaScript.

The CGI program is just as simple: Read the form data, process it, and return the HTML page. The only oddity is that the arguments have been collected into a %args hash before use. In this simple case, that’s unnecessary, but the point is that all the form data can be put into one Perl data structure.

Let’s examine this form submission from a protocol point of view. An HTTP POST request carries the form data to the server. The form data consists of name=value pairs. An HTTP response carries back the replacement Web page. The Web page is a document of any type, in this case type “text/html.”

None of the protocol content is very standard. The request content (form data) just contains whatever the NAME attributes of the form elements are. If the form elements change, the submitted data changes to match—the two are linked. Also, you can’t send two pieces of data with the same name without confusion, so it’s inflexible. The returned document, on the other hand, is too flexible; it can contain any rubbish.

It’s not a very clean messaging system either. The page that sent the original message gets changed, instead of just sending and receiving what it needs. Perl can return nothing (a 204 ‘No Content’ response), but then the response can’t carry a message back to the browser. From a pure messaging point of view, it’s all a bit unstructured with too many side effects.

Reworked submission with SOAP-isms
Consider the reworked Simple.html, now called Complex.html (Listing C), with two frames, Hidden.html (Listing D) and Visible.html (Listing E). The Simple.cgi is now Complex.cgi. It’s still pretty simple. The service to the end user is unchanged. All that has happened is that the messages to and from the server have been cleaned up to mimic SOAP.

Some changes are necessary but trivial. Simple.html (Listing C) now has two forms. Form 1 (Listing D) has all the text fields as before. Form 2 (Listing E) has the Submit button and a hidden field. When Form 2 submits, any response will be loaded into the “fhidden” frame. In fact, it will be a variation on the initial Hidden.html. This allows Visible.html to sit before the viewer without disappearing. That is SOAP-like. SOAP does its messaging behind the scenes and doesn’t replace anything.

There are two JavaScript functions, send_request() and get_response(). Let’s consider send_request(). Since Form 2 has only one field, all the submitted data must be packed into it. The default approach of name=value pairs won’t work anymore because there’s only one pair. Send_request() packs all the data into a structure. This structure can be as complex and as nested as needed, since we’re free to make it up. The way it’s done here is to create a bit of Perl. The data is a Perl structured literal. Although the names in the literal match fields on Form 1, this is now just a programming choice. This literal is sent to the server on form submission.

In Listing F (Complex.cgi), the sent Perl data structure is dug out. No matter what it contains (as long as it’s a Perl literal), the magic eval() line will extract that data. That is very SOAP-like—the server can handle any structured data. How it handles the data depends on whether it recognizes the fields in the structure. In this simple example, Complex.cgi can work only if the content is understood. In SOAP terms, the request implies that mustUnderstand is set to 1 (true). The server must understand the message, or it will fail. In real SOAP, you have to set this flag if the understanding constraint is to apply.

Complex.cgi sends back a response, which is loaded into frame “fhidden.” The response message is an HTML page (to make the example work), but the essence of the response is another data structure: a JavaScript object literal. This is where get_response() comes in. SetInterval() has been calling this function every 100 ms since Visible.html first loaded. When the response flag in the structure is true, get_response() grabs the object literal and puts the answer data into Form 1 for you to see—job done. This is the same as the Perl case, but in reverse. The browser mustUnderstand the response, and the response is structured data, this time made of JavaScript instead of Perl.

So structured data goes both ways, and each end must understand the structure that’s received. Neither end is necessarily reloaded (the CGI could be in mod_perl). The structured data must be in a language that the recipient understands (Perl on the server, JavaScript on the client). The price paid is that code on both sides is required to pack and unpack the structured data (one-liners on the server, in this case), and if one end expects a response to its request, it must wait and wait for it.

Stepping into SOAP territory
Now, let’s SOAP-ify the example. When the request is sent, it is in one form field called request. In SOAP, this is called the envelope—the outside boundary of the sent message. So use a SOAP envelope instead of a one-field form. An envelope is just an <envelope> content </envelope> tag pair.

If you had PHP on the server instead of Perl, both messages could use the same syntax for their content (JavaScript and PHP happen to share some syntax). Then, you would have the same message syntax both ways. But you could go further and use XML as a standard syntax instead. Browsers support XML data islands now, and servers have XML libraries, so both ends can read the XML. Pick any or all of the numerous XML standards as a starting point. That’s the SOAP way.

Next, let’s reduce the JavaScript effort. We’ll upgrade the browser so it has SOAP support (like Mozilla 1.0 / Netscape 7.0). This means that there are handy functions available for packing and unpacking the data. You might not have to write them. Call this marshalling and unmarshalling from now on. These functions are a bit like using escape() to get a URL or a cookie right. Maybe they’re done with XSLT. The upgraded browser might also have a SOAP request function separate from the normal form submit(). If so, you don’t have to worry about the page being replaced. This submit function will wait until the response comes back, so setInterval() isn’t necessary, either. In short, a SOAP-enabled browser provides the interface you need.

Download the files covered in this article

The files are available in one zip file:

Next, you have to get agreement on who mustUnderstand what. So, every time a message is sent, include in it extra data—a URL to a resource that describes the message’s format. Perhaps that resource is a DTD or an XScheme document. Now the recipient always knows what it is that must be understood without guessing. Then the recipient can say with total confidence, I don’t know what to do (or more likely, I do know what to do).

Finally, if you want, you can take away the browser. Just use two XML server programs talking to each other for business-to-business reasons. Keep the use of the HTTP request/response pairs, though, because SOAP often relies on them to do the grunt work even when no browser is involved.

Lather up!
SOAP is a protocol for passing messages. The most common arrangement for SOAP messages is a request/response pair, basically a question and answer. HTML form submission is also a request-response pair. SOAP is to form submission a bit like Microsoft Windows is to DOS. DOS does a really basic job, which is great if that’s all you want. Windows is more complicated, but once you’ve learned the detail, it’s more powerful too. You can mimic SOAP concepts with plain HTML forms, but for the real thing, learn the SOAP syntax and buy some three-phase power tools.