Software Development: Integrating large blocks of code into a single project

Large application development projects often involve the integration of disparate code blocks. Decisions on how to proceed in such situations are determined by numerous factors, which are explored and explained in this overview.

Working on large projects isn't like working on small projects for a variety of reasons. Among these is that when you're working on large projects, you're often working with large blocks of existing code, called packages, which need to come together. In smaller projects you can sometimes create your own solutions for just about everything. With larger projects, however, come larger chances that you'll be buying pieces of the solution and trying to glue them together.

Boat + Car = Duck
If you've never been to the Wisconsin Dells, you're missing something. In addition to the natural beauty, there's an oddity that I can honestly say I've never seen anywhere else. When you're in an area where there are countless lakes separated from each other by small bits of land and you want to get around, you need something that works like a car and works like a boat.

You need to be able to roll your way across the land and settle into the water ready to power your way across. However, neither a car nor a boat could handle both tasks well. That's why in the Wisconsin dells they have vehicles affectionately known as "Ducks." These inventions roll across the ground like a car and float through the water like a boat.

What's so amazing about this is that most people wouldn't think that a car and a boat could be successfully merged into one vehicle, yet that is exactly what was done. Merging those transportation technologies is similar to some of the feats that developers are asked to pull off when integrating two very different systems.

Very different blocks of code
One of the challenges that people have with larger projects is that often two very different approaches to the same problem are taken by different blocks of code, and those different approaches need to be integrated. One package might be running J2EE on a Web Sphere server, and the other package might be running ASP.NET on a Microsoft IIS6 server. Both solutions may have good features and meet most of the needs of a subset of users, but the trick is taking two potentially large packages of incompatible code and making them work together.

In some ways, this programming task is as difficult as making car and boat components work together so that you can navigate both land and water with ease. In both cases, the project requires design decisions that compromise between the best features of each to come to one solution. In the case of programming, the fit between two different blocks of code is difficult because in some ways the two (or more) solutions will surely overlap. Each block of code will solve the same problem—invariably in a different way. In other cases, there may be a gap between the blocks of code that isn't covered by either.

The trick is determining which of those blocks of code will be modified to support or accept the other's way of doing things. Converter code may have to be written that ties one of the products into the other. Determining where those integration points should be made isn't always as clear, and the decision often rests on many factors.

Deciding on a direction
There are two different kinds of integrations. Where there is overlapping functionality, it's necessary to determine how to make one set of functionality behave like the other. In the other case, where there are gaps between the prepackaged solutions, the question becomes which solution you extend to cover the gap. Evaluating a few key aspects of the desired solution will help you nail down an integration strategy.

Future direction
Perhaps the most powerful factor that will influence which packaged solution you'll choose to enhance and extend will be the future direction of the organization. With this knowledge, you can determine what resources will be available to maintain the chosen solution. The part of the code that you modify is where the most support will be necessary and therefore should, if possible, be aligned with the future direction of the organization.

Of course, this assumes that your organization knows what its direction will be. The big Sun/IBM and Java or Microsoft and .NET decision makes a great start. From there you can decide on database platforms, server platforms, and a list of other technology directions. The beauty is that making the list is as easy as reading a few reports, understanding the business, and talking to a few references. Once you make the list of the technologies that you want to support, you will have essentially built your technology vision, which is useful well beyond just integrating some code.

Technical knowledge
The second most telling factor in determining which package to extend or modify is the current technical knowledge of the group. In particularly thorny application integrations, decisions can be driven by the resources that are available. It's the old cliche: "If life gives you lemons, make lemonade." If you know how to modify one packaged solution but not another, it's likely that you'll need to make the decision to modify the one you know.

However, the advantage of current technical know-how must be tempered with the long-term vision. If you're going to be doing .NET but all you have is Power Builder application developers, it may make sense to invest in some training and work on moving the skill set of the staff toward the intended platform. While this process can be much more difficult in the short term, it can lead to a much better long-term solution.

Once an organization has committed to a project, they generally want it as soon as possible—or sooner. So despite the best intentions of laying out an architecture that supports the future, it may be necessary to put most of the integration work into those components which can be modified the most expediently. This may mean modifying a solution that you don't want to, perhaps putting more work into a solution that won't ultimately be the best solution. However, it's sometimes the right decision.

The final major consideration for how two large blocks of code will come together is which pre-made solution is the most extensible. If one solution is easily extended and modified, it makes sense that you would want to make your modifications to that program.

The unfortunate thing about integrating large blocks of code together is that rarely is there a single elegant answer which fits perfectly within all four of the preceding criteria. Unfortunately, compromise is the name of the game. The trick is to make the right compromise between the "right" solution and the immediate solution—between the long-term goals and the short-term goals.

Editor's Picks