When maintaining software systems, especially for a large client, one of the major concerns is backward compatibility between releases or updates. However, control of what is and what isn’t compatible seems to be shifting from the developers to the vendors. Because of this, you must be even more vigilant over your entire development environment.
I define backward compatibility as the degree to which a software update does not negatively affect the user or client. In the early days of software development, a computer program was a largely independent module, and decisions regarding the costs and benefits of maintaining compatibility between versions were entirely in the hands of the developers. As more and more features were added to operating systems and the use of third-party libraries, components, and modules became widespread, these choices moved from the developers of the application to the suppliers of the operating system and other modules. And what is cost-effective to these suppliers may not be cost-effective to the developers of the deliverable application.
Here’s a cautionary tale that illustrates the effect this trend can have on development projects.
Problems during development
The project I’m currently involved in has been ongoing for four years. It is a three-tier design consisting of a Visual Basic client application running on Windows 95, a Windows NT middle tier developed in Visual C++ and Visual Basic, and an Oracle database for the back end. The project was initially developed under Visual Studio 5, and several minor updates were also released. For the next significant update, we migrated development to Visual Studio 6, and we have just released a minor patch under this environment.
Our first brush with lack of backward compatibility issues came toward the end of our initial major release under Visual Studio 5. Our client requested a metric listing the number of lines of code in each software module we developed. One of our developers found a Visual Basic analysis program, so we ordered it, installed it, and found it quite helpful in creating a set of software metrics for our client. Afterward, however, we began experiencing intermittent failures on our test client. The developers would retest the software on their development machines, and all went well. Finally, someone would simply rebuild the application, and it would run without problems. After several iterations, we realized that software built on the machine with the analyzer program would not operate on our test client machine. Uninstalling the program did not resolve the problem, so, as we were at the end of the development cycle, we simply got a yellow Post-It note and labeled the machine "Do Not Use for Production Builds."
After delivery, one of our C++ developers began evaluating Visual Studio 6. Upon his first attempt to build our released production code, one C++ program generated several hundred compiler errors. After reinstallation of Visual Studio 5, the problem persisted. Rather than endure the loss of two development machines, we decided to move all developers to Visual Studio 6. However, until we deployed the next version, we still needed to be able to maintain and patch the Visual Studio 5 version. Therefore, we installed Visual Studio 5 on our test client and used another yellow Post-It note stating "Version 1.0.2 Maintenance—Do Not Use." We no longer had a test client and had to rely on testing from development machines.
After two weeks of effort, our C++ modules compiled again. Specifically, the problems stemmed from two Microsoft library function call changes, one in the ATL library (the Active Template Library, the basis of Microsoft COM) and one in the ADO library (the ActiveX Database Object library for database access). We also reviewed the more than 60 files in our install set and determined that over half of them would need to be changed to a newer version.
The Visual Basic conversion went largely without incident, but a significant amount of effort went into producing code that had no functional changes from the previous version—and we had lost our test client machine. Toward the end of the new development effort, we were able to obtain a new test client, but to this day, our previous test client sits in a corner, with its yellow Post-It note curling at the corners, unused.
More problems—after release
Our latest bout with backward compatibility occurred just prior to the writing of this article. After some strong user reaction to a feature in our latest release, we prepared a quick-turnaround patch to address the problem. After what we believed was a thorough retesting of the application, we shipped the patch. Within a week, we started receiving help desk calls about a catastrophic failure in a section of the code that had been untouched by the change. The function was simple and had long been part of the operational code. It merely saved some user-specific configuration text to a file and read it back later. As it turns out, the system on which we did our final build had been upgraded to Internet Explorer 5.5. This change had updated a system file, Scrrun.dll, which is used by Visual Basic for some file operations. Although we had made no functional changes to the code whatsoever, our new patch now looked for the updated version of this file and failed if it was not present. We were required to issue a patch to our patch.
For our more supportive users, this really did not pose a problem. But it gave our more critical users more room to question our project. We had gone from the positive of reacting quickly to user requests to being viewed as unable to get even a simple change right.
What did all this teach us? First, in the Microsoft world, it is virtually impossible to maintain control of your development environment. Either request that your client or development department provide a dedicated machine with a full development license for release builds or set up a mechanism to track what is installed on your development machines. We have taken the latter approach and have written an automated program to verify the system files installed on a machine against the set specified in our install program.
Second, delivery of an executable image only is not possible. Expect every customer delivery to require a full install program and expect to have to upgrade the system files in the install program for every release.
Third, a manual regression test is not feasible for any major software program. We are currently investigating mechanisms to automate our testing effort, including the addition of testing modes and features.
Fourth, for us at least, Visual Studio .Net can wait. I have instructed our developers not to install any of the Visual Studio .Net beta CDs freely available from Microsoft. We simply cannot afford the loss of a development machine nor time to investigate a system-wide upgrade for our development team and user sites.
How to check versions
Here are some resources for you to do version checking:
- For basic version checking: From Explorer or Find/Search, right-click on a filename, select Properties, and then select the Version tab.
- For advanced version and dependency checking: Use the Depends.exe utility available with either Visual Studio or the WindowsNT Resource Kit. The first time you run the program, select View, then choose Configure External Viewer. This will make Depends.exe available from a right-click. Many times, Depends.exe will give a more accurate file version number than the Properties method above. This utility will also list DLL dependencies and flag dependency problems.
- For programmatic version checking: See the documentation for GetFileVersionInfo() under Visual Studio help.
- To determine the source of various file version numbers: Go to http://msdn.microsoft.com/default.asp, select Resources, and then select DLL Help Database. Using this database, you can look up any Windows system file, find all the released version numbers, and determine what Microsoft applications may have installed a particular version number and what distribution sets are available if you need to apply this change.
For developers, the advantages of reusing third-party software are tempered by the costs of maintaining compatibility. The major developers often place less emphasis on backward compatibility than on the addition of new functionality. This is particularly vexing in the Microsoft world, where any and all applications are free to update system files, making the concept of a "version" of an "operating system" meaningless. In today's environment, it is the responsibility of the development team to constantly monitor for and react to changes in the development system environment in order to ensure the release of operational code to their users.