System testing encompasses many types of tests, which can include performance testing, stress testing, security testing, requirements testing, documentation testing, usability testing, and training testing. Our focus here will be on three additional system test categories: disaster recovery, multisite, and interface testing.
Did you miss it?
We’ve covered a lot of ground in our series on software testing. For more information, check out these articles:
- “Keep the software testing life cycle spinning”
- “Think ahead: Defining your testing strategy”
- “Hammer out your tactical testing decisions with a Testing Plan”
- “Unit testing provides your first line of defense against application flaws”
- “Integration testing will show you how well your modules get along”
- “System testing: Can your program take a lickin' and keep on tickin'?”
- “System testing to check security and validate system requirements”
Disaster recovery testing
A computer environment is subject to many physical and natural threats. Even though your application may have a high degree of security built in, it may still be susceptible to a disaster that affects your entire environment. These vulnerabilities range from terrorist threats to floods to fires to computer viruses to malicious damage caused by employees. What would happen to your shiny new application if a fire destroyed the computer center? Yes, the system tapes can be recovered, but can your application be recovered successfully as well? I can picture your heads nodding yes, but I wouldn’t be so confident. When would you like to find out for sure how recoverable your application is—when a disaster hits or during system testing?
Disaster recovery testing is designed to see whether your application can be recovered successfully in an alternate environment. The procedures you establish will be the basis for periodic disaster recovery testing throughout the life of the system. You want to test for these things:
- Can the environment and operating systems be recovered in a reasonable time period?
- Can all application databases be recovered to the same point in time, thus maintaining consistency between the databases?
- Are processes in place to recover transactions from the time that the backup tapes were created up to the current time? For instance, if a disaster occurs on Thursday, but your offsite backup tapes are from the previous Friday, can you recover all of the transactions from Friday to today?
- Does all of your security work on the new hardware? Are there server-specific certificates that will not work on the recovery servers?
- Are there documented procedures in place to ensure that the support staff can recover the system? Remember, they may or may not have the same level of knowledge that the development team has.
- If databases and files are needed from other applications or vendors, are they also recoverable, and will they sync up with the recovered application?
Obviously, disaster recovery is a specialized test. If your application absolutely must be available, you probably want to do a form of this test even if it is a simulated instead of a live recovery.
Multisite testing is designed for applications that are running on multiple sites and that need to communicate with each other. The test is set up based on how the sites communicate and what dependencies exist between them. If the sites run independently, this test is easier than if they share dependencies. Here’s what to look for:
- How do you recover if one site goes down? Can the site recover locally? Do you need to push the recovery files out over the network?
- Can you get all of the sites back in sync if one goes down?
- What complexities are associated with two or more sites going down at the same time?
- Can each site be recovered if a disaster occurs? This has similar implications to the disaster recovery test explained previously.
Depending on the architecture of your application, multisite testing may be crucial. The more sites you have that run the application, the higher the chances are that there will be a problem somewhere. This test can also be difficult to run unless you have a test environment where you can simulate multiple sites or unless the application is already available on multiple sites. If you have a staggered rollout schedule, portions of this test may need to be run as each location is ready to go live.
By this time in the testing process, you may feel that you have tested the application to death. In fact, as early as integration testing, you may have been testing the interfaces to your application. This should especially be the case for files and databases that come from other internal applications. However, this interface test looks specifically at all of the system interfaces, especially those from outside vendors or suppliers. The test should address these issues:
- Have you processed vendor feeds exactly as they will happen in production? If not, do it now.
- Have vendors and suppliers who receive data from your application thoroughly tested the resulting outputs? It is not enough to see that a file for a vendor was created. The vendor must process the file and ensure that the results are exactly as expected.
- Have you validated that the timing of the interfaces is as it should be? If the file you create for a vendor is perfect but is available too late, you may still have problems.
- If there is input and output from an interface, have you tested both ways?
Your prior testing should ensure that your application is accurate, usable, and stable. Interface testing makes sure that there are no surprises that arise from areas outside of your application.
Consider these points as you build your system test suite:
- Disaster recovery testing ensures that your application can be recovered successfully in case of a disaster. This should at least be considered in a paper walk-through if you can’t simulate a real disaster scenario.
- Multisite testing is absolutely necessary if you have multiple sites that communicate and need to remain in sync. If you don’t test now, you will be in for unpleasant surprises when the application goes live.
- Interface testing catches all interfaces that have not been exercised already. This is an opportunity to test with external suppliers or vendors.
It may seem as though after system testing, there is nothing left to test. That’s close, but one more important test remains. Customers should not sign off on the application until they’ve had a chance to do the final testing themselves. If your prior testing was thorough, this is a no-brainer. Unfortunately, there are some problems that make it through all the prior tests. Our next article will look at user acceptance testing.
Project management veteran Tom Mochal is director of internal development at a software company in Atlanta. Most recently, he worked for the Coca-Cola Company, where he was responsible for deploying, training, and coaching the IS division on project management and life-cycle skills. He's also worked for Eastman Kodak and Cap Gemini America and has developed a project management methodology called TenStep.