You can execute extensive tests and provide reports to determine performance levels and the breaking points of an application with many available tools. Like most tools, you get out of these what you put into them and you must first thoroughly prepare for the tests. I’ll cover five tasks to consider before you execute a formal load test.
Tune and tune again
A load test consultant I recently worked with stressed that the biggest problem he encounters is customers who don’t know the difference between functional and load testing. I assured him that we were different and that we had in fact completed functional testing and were ready to load test. One small demo with the consultant’s tool proved that we weren’t ready to begin testing with multiple users.
In functional testing, it’s easy to focus only on the use case or the completion of a function and not on the individual performance of the test case. Before you can even get to testing concurrency or stress, each single user case must be optimized. Our gap was in testing from a single user perspective rather than looking at concurrent users.
Tuning is an iterative process, but a thorough effort completed prior to gathering resources and spending hours on the load test will prevent you from having to complete too many cycles.
Agree on performance goals
When you sit down to plan the testing with representatives from various areas of the business, you may be surprised at the ideas everyone has about performance and the possible load the business may predict. You’ll want to discuss response times, hits per second, acceptable error levels, and expected numbers of concurrent users.
While it may confuse nontechnical management, it is important not to get caught up on the numbers, but to complete the main performance goals of a load test: stressing the application, testing the ability to support concurrent users, and monitoring performance as well as setting benchmarks that can be used to gauge future levels.
One concept that is important for load test measurement is the virtual user. Virtual users are the load test tool’s users that simulate activity on your application. Depending on your load tool and the script setting, it is not always easy to draw an equivalent value between virtual users and actual users. Also, in a simulation, 100 virtual users may perform an activity that 100 real users could not or would not perform.
Some tools will let you set a delay time to represent the time that a user will spend on each screen prior to clicking or creating an event that changes screens. Or, you can perform the scripts without any delay to represent super users that run through transactions even faster than a group of programmers who have drunk too much coffee.
Stabilize and prepare the environment
Make sure you’ll be performing the load test on a level playing field in which environmental, hardware, and software properties aren’t changing during the test period. Setting performance levels and benchmarks requires control of the factors that produce those numbers, which makes the change control process even more important during this time.
In addition to controlling change, you must make sure the environment is ready for the load that will be placed on it by the test. This seems like an obvious requirement, but it is possible that the load test will hit volumes not expected even under normal production situations and will accumulate those levels in a shorter period of time. For us, this meant experiencing a higher bandwidth usage on our burstable Internet pipe and exceeding the planned capacity of activity logging in our database. All of the people responsible for components of the application must be aware of the load volumes that will be generated during the testing.
Gather ownership from all of the development units
The load test, as a test of the integration of all the independent tasks performed in different business areas, will require cooperation from all sides to determine points of failure. You’ll also need a bit of political savvy to ensure that sides aren’t deflecting blame, pointing fingers at other units, or perhaps even worse, not paying attention to the load test.
As the application is being built, each area of the business will be focused on completing its individual tasks, such as coding business logic, designing HTML or XSLT presentation, or building network infrastructure like firewalls or other authentication components. To keep the test moving along, each area should have a representative available for debugging and making changes. If one of these areas is allowed to defer this responsibility, the test will stall or fail.
Prepare for the reiterative test process
To some extent, you never can complete load testing. Because of the dynamic nature of Internet applications and hardware, many factors inevitably lead to a constantly changing level of performance. During the early stages of load test planning, take steps to ensure that the tests can be repeated and that benchmarks are recorded so that you can compare the results with the results of previous tests.
You’ll also want to define procedures or strategies in cases of test failure or real-world performance failure. The plan should include how the problems will be escalated, resolved, and communicated to stakeholders. It should also include contingencies for stopping the test and resuming it once bottlenecks have been resolved. The groundwork you do during the initial stages of load testing will set the team up for reiterative success.