The reason consultants are brought in to do integration work is because it's extremely messy, extremely difficult, and extremely thankless. "Integration" usually means preserving legacy applications that are cumbersome and often inexplicably sacred to your client, squeezing miracles out of the none-too-clean data they yield, and trying to bring up faster, cleaner apps from an enterprise resource planning (ERP) source you're learning on the fly. You didn't design it, but yours is the head that will roll if the whole thing doesn't absolutely sing.
The word middleware covers a great deal of territory. In general, it defines a class of integration software that governs communications between components in an application system. Specifically, it's used to monitor and route intercomponent requests and manage the app system's runtime environment. It can be found in any number of places, and for each form of intrasystem request, there's a specific class of middleware. Here are the most common varieties you'll encounter:
- Distributed processing—This middleware handles interfaces in the application system, coordinating requests between components. It basically enables app components to talk to each other, imparting fine control over both event handling in the application and the dynamic disposition of data.
- Database access—This middleware sits between applications and data, and sorts out what data goes where and under what conditions; it's not just about databases, but can also include conventional file I/O. It handles request management and can oversee database connectors.
- Remote procedure calls—This old friend is one you've used many times before, though perhaps not in this context. RPCs are the mainstay for initiating application events across a network. The concept is to run programs in an application system transparently, giving the system a level of process control that requires no user intervention. While they've been around a lot longer than Enterprise Application Integration has, their usefulness here is obvious.
Middleware is intended to save you bargeloads of time you'd otherwise waste in rewriting legacy apps and reconfiguring databases. But more than a few IT professionals shout curses rather than sing praises when middleware needs to be put to work. Why is this?
The hard part
The point of application integration is to preserve your investment in existing applications. The reason integration is required is that you no longer operate in a world where users are people sitting in front of terminals. Users, from the legacy app's standpoint, can now include those original users plus downline enterprise apps, lateral apps producing similar but enhanced data, Internet users, Internet apps, and many other destinations.
On top of this, a legacy app's input can now include data that is similar to what it was designed for (multiple databases, real-time queries, remote real-time queries), and worst, these inputs can become recursive.
In short, when you wrap middleware around legacy apps, you must deal with much more than we typically have to handle in writing an app in the first place. You might easily ask yourself: Why not just rewrite the legacy apps?
What lies beneath
The power of application integration seems almost serendipitous, and we tend to get tangled in the technical side and overlook the incredible benefit that accommodation of legacy apps has bestowed upon us. Because industry efforts to accommodate legacy system investment influenced enterprise development early on, a typical application integration framework operates on the principle of stateless messaging.
Applications no longer necessarily operate in linear series. Instead, apps generate data, which becomes generally available to other lateral or downline apps. Via messaging, those apps may then make use of it as needed. Bodies of data become free agents, in a sense, and new mechanisms for the initiation and regulation of app execution are now in place, giving us tremendous discretionary powers over processes as a whole and even redefining the role of the individual application in a business process.
All of this now rests in your hands. Most books and articles will coach you on the techniques by which you can implement data queuing, messaging, and application execution security, but there are broader implications to these abilities. You can do more than streamline processes with this level of control. Not only can you greatly enhance effectiveness, but you can have unprecedented control over departmental functionality and interdepartmental cooperation. You can also extend to interactions with partner companies, as well.
Data portability increases dramatically via middleware
In most legacy application systems—and in many contemporary distributed systems—transactional data exists as an interim step in a linear process. While data-queuing and interapplication messaging were originally created to bring legacy app data together with enterprise app data, the truth is that middleware "liberates" transactional data regardless of source and destination. Transactional data now becomes available to all.
Opening databases that were originally department-specific is obviously a key step in ERP in that it more deeply integrates the awareness of your client's various users. It's a mistake to overlook the possible advantages of making transactional data available across functional boundaries because live, momentary, real-time data can hold critical process cues that may never have been evident before. It's important to understand that these new free-standing pools of fresh information, made available beyond the legacy system's original destination, can often be useful in ways you never thought of.
Passively monitor transactions and implement a "super app"
Component transaction monitoring (CTM) has been with us for some time. Microsoft would love to claim it, but IBM was really the first player at the table. Transaction monitoring allows you (via real-time middleware) to keep an eye on a program, a file, a database table, or any other system component. Then, depending on the sophistication of the middleware you're using, you can set up conditional responses to changes in that component—the sending of e-mails, the setting of flags, the storage of messages in your app integration messaging system, the opening of permissions for program executions, etc. Basically, you can now write a "super app" to surround individual apps and even impart a new layer of control and sophistication on an existing system, increasing its efficiency, availability, robustness, or all three.
By combining transaction monitoring of a legacy app with the messaging utility of middleware, you effectively give the app the power to communicate with other apps. Sensing an instance of a program's use, identifying the user, and noting the data going in and out basically give you the power to fully report on the app's activity to the middleware message pool. In this way, programs can "talk" to each other, and no distinction is needed between legacy apps and full-blown enterprise apps—they can now be on equal footing.
What does this mean? Programs that contribute to a business process previously relied upon their position in a linear process chain to function, but they can now operate dynamically (read "over the Web") without having to be rewritten. And here's the real punch: Since the value of the very pricey application software suites that your ERP provider wants to sell you operate in this manner, very often legacy apps can function with the same dynamic utility of expensive ERP counterparts, given the right middleware configuration.
Restructure processes without revising the applications
Because you can impose conditional responses to any program activity based on the content of the reporting, you can reconfigure the individual steps of an application system into new, more powerful processes, affecting any or all of the following:
- Precision of event timing in the process
- Dynamic levels of user access, enhancing not only security but situational appropriateness of user interaction with the system
- Fine resolution of reporting, down to real time
- Enforcement of critically timed decision making
- Timeliness of decision support data
- Efficiency of planning through detection of errors (in real time) at a finer level
- Quality of data
- Higher degrees of communication and awareness between users and departments
- Much greater contingency handling in business processes
- Isolation of human decision points throughout a business process
- Empowerment of users at the points in a process (and only at those points, if desired) when empowerment is appropriate
- Precise enforcement of process requirements from users
- Greatly enhanced process management overall
And you don't have to touch a single line of legacy code or enterprise app code to accomplish these things. It's no longer about using middleware to reach predefined goals. You can use middleware to leapfrog your thinking to higher, more effective goals.
Scott Robinson is a 20-year IT veteran with extensive experience in business intelligence and systems integration. An enterprise architect with a background in social psychology, he frequently consults and lectures on analytics, business intelligence and social informatics, primarily in the health care and HR industries.