One of the mixed blessings of BizTalk
Server 2004
is the orchestration feature, a graphic process configuration
tool with few equals. Incredibly flexible and exceedingly well-suited to
business process design and integration, its correct usage is unfortunately
also opaque in many applications and woefully under-documented (so far, anyway—we’ll
keep hoping!)

Another positive of BizTalk Server is the BizTalk Explorer
side, where all the limbs of BizTalk flex and extend. Don’t fail to appreciate
this layout. The clean BTS separation between business logic (nested in
orchestrations) and interface/database (in Explorer) isn’t particularly
conspicuous, since it’s possible to do just about everything in orchestrations.
But it’s a separation you should keep in the forefront of your designer’s mind,
because it can buy you a lot.

Roll your own rollback

Here’s a basic design principle that will often be
applicable and perhaps appropriate to your particular distributed/integrated

acknowledge and format the data shared with partners in BizTalk Explorer;
unleash the data on your business logic and internal messaging in

Some of the BizTalk tutorials, samples, and labs you’ll come
across will employ this principle implicitly. Others will go another direction.
Don’t take either course blindly; instead, give careful thought to when (and in
what form) you’re going to need to handle your data as it makes its way through
a process, and design accordingly.

Why? Because BizTalk Server 2004, sadly, does not offer
rollback; its Health and Activity Tracking utility doesn’t always permit you to
resume the handling of a message; and BizTalk Explorer robustly facilitates the
features of your process design that aren’t business logic-oriented, making
your overall implementation effort faster and easier to debug.

Isolating business logic in orchestrations and leaving the
pitch-and-catch of data in various stages of dress in Explorer permits you to
temporarily store data at any point in a process that you might want to do so.
You can effectively create de facto queues for transitory data in a matter of
minutes (even in a solution already deployed in production), where it can be captured
for review or troubleshooting, copied and forwarded to need-to-know parties or
ancillary processes, or sit patiently until rescued if something goes haywire

What goes where, and why

In a typical distributed business process, there are various
transitions data usually has to pass through. It is received from some external
source in some foreign format; it is acknowledged and interpreted; it is
transformed into a friendly format; it is validated and parsed; it is analyzed;
it is introduced into a business application, or stored for later application;
it is culled for transitory reports; it is handed off (in modified form) to
other down-line applications or to external partners. Along the way, people and
other systems are often made aware of the progress of the data at any of these
points. This scenario has many variations, but whatever the order of events,
the steps above are fairly typical.

If you apply the principle presented above, then most of
these steps will occur in BizTalk Explorer, rather than in orchestrations.
Specifically, the only exceptions would be the introduction of data into a
business process, and what is done with the data within that process. The
creators of BizTalk might not be crazy about such a slim application of the orchestration
feature (remember that just about all of these steps can be done in an
orchestration, if you choose that route), but there are advantages to this
segregated design approach:

  • When
    a data-related error knocks an orchestration off kilter, it’s often
    difficult to locate the error message, let alone correctly interpret it;
    data-related errors in BizTalk Explorer are more easily interpreted;
  • A
    message that contains data, lost or suspended in an orchestration
    exception, may or may not be recoverable; in BizTalk Explorer, the data
    can be retained at various points, making the process restartable from
    those points even if the message is “officially” suspended;
  • Anyone
    can see and understand what’s going on in the BizTalk Explorer portion of
    a process, since it’s basically just messages with date/time stamps making
    their way through a series of folders; BizTalk orchestrations, while easy
    to follow, nonetheless require considerably more effort to decrypt.

In a general scenario, a purchase order is received from an
outside partner via TCP; a BizTalk pipeline catches it at the receive port,
validating it and identifying the sending party, then returning a PO
acknowledgment; the purchase order is re-mapped to an XML document; the XML
document is parsed and transformed down to purchase order elements that are
.NET typed, and the record is finally stored in a SQL PO table.

In our example, we’re using BizTalk Explorer’s send/receive
adapter features to filter the inbound data and make decisions about storing it
(in different stages) based on that filtering. In this particular scenario,
we’re archiving both the inbound order and the acknowledgment that went back;
we’re using a hot folder to pass the XML version of the purchase order into the
orchestration and our archives function as a log that tells us not only what
happened on our side, but what we told the sending partner.

Why? Because if something has gone wrong and the
orchestration portion of the process is down, the purchase order will sit
quietly in the folder until the process resumes. We don’t have to back up any
further in the process to restart; we have a time stamp on the interim version
of the order that tells us when things went awry; and we can leave the BizTalk
Explorer portion of the process running while we fix whatever’s wrong, knowing
that purchase orders will safely accumulate in the to-be-processed folder while
we work.

Nuts and bolts

Finally, here are the nuts and bolts of the BizTalk Explorer
side. There are several tricks you need to know:

Filter inbound data by sending party and document type

You’d perhaps conclude intuitively that you’d do this
filtering in your Receive Port Location, but in fact where you want it is in
the Send Port filtering. Open BizTalk Explorer from Visual Studio.NET View, and
open up the Send Ports under the BizTalk configuration database (.BizTalkMgmtDb.dbo).

Adding one for the first time? Right-click on Send Ports to
bring up the Add Send Port wizard. Right-click to Edit the port. Here’s the
thing to keep in mind: you can have multiple Send Ports for any particular
message-receive from a partner (called Party in BizTalk Explorer), as you’ll
see below. Give each Send Port you create for a particular partner/process a
name that makes clear what you’re doing, like MyPartnerAcks, or MyPartnerArchive,

When editing the Send Port, go to Configurations | Filters
& Maps | Filters. Here, you can parse messages based on type (BTS.MessageType, for instance), to sort
out acknowledgments, parse header data from detail data in multipart documents,
and so on. You can combine this with filtering on the ID of the sending partner
(based on a field in the header, perhaps) to route inbound data to the correct
folder (see Figure A).

Figure A

Route inbound data

Use “file” Transport Type liberally

When setting up multiple Send Ports for a particular partner
or process, creating a “copy” of the data is almost an effortless
exercise. Create a Send Port specifically for this task, and select FILE for
Transport Type in Add Send Port. You’ll need a URI, a location and filename (Figure B). For the filename, the
default is %MessageID%.xml. Change
this to .txt.

Figure B

Add Send Port

Set up trading partners (Party) for multiple Sends; or send data to
multiple destinations with Send Port Group

Parties in BizTalk are partners, identifiers for business
entities with whom your process will exchange data. Some industry-specific
accelerator packages, such as HL7, for BTS2K4 allow you to configure Parties
for multiple Send Ports; that is, a batch of data arriving from that Party
will, upon being identified from the message header, be sent to all the Send
Ports for which that Party is configured. Alternately, you can set up a Send
Port Group (Figure C), which will
give you basically the same thing—a collection of Send Ports, bound to
different destinations, filtering on a common condition or conditions.

Figure C

Send Port Group

Use the appropriate pipeline when capturing data

The Send pipeline you choose when setting up a Send Port (Figure D) determines the format of the
data when it lands in the file you’ve sent it to.

Figure D

Send pipeline

Your choice of pipeline depends on what you want done with
the data when the file lands in the destination folder. That is, if it’s to be
passed another process, you’ll want it in the form the process is expecting
(often this will be XML). If you’re setting up a “hold” folder where
you want data stored for “rollback” at some sensitive point in a
process, then you’d likewise want it in the format it would need to be in when
the process restarts (again, often XML). If you want to archive the data, or
pass it to a person for review or reporting purposes, you may want it in its
original form (often simple text).

Whether you’re using canned or custom pipelines, it’s
important to make the right choice for the right job, as you configure your
Send Ports. Among the default BizTalk pipelines, for instance, the following
pipelines yield the following results:

  • Microsoft.BizTalk.DefaultPipelines.PassThruTransmit
    passes a message in the format it’s already in (i.e., no post processing
    occurs in the “send” version of the pass-thru pipeline)
  • Microsoft.BizTalk.DefaultPipelines.XMLTransmit
    delivers XML

Then again

The practice of separating interface/database processes from
business logic is usually best, but there may be reasons why you need an entire
process situated within an orchestration. When there’s good reason to do so, by
all means, take this course. We’ll look at some scenarios where it is indeed
the best practice, and some tips on how to organize such an orchestration, in
an upcoming article.