Discussion Locked

Behind the Scenes

0 helpful
291 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

This is page 3 of the discussion. Read the opening post

291 replies

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the real world and smoothly transition into live service.

Wrong approach! This is extremely risky and misses out on the many benefits of starting load testing early and applying it throughout the project. With this approach does the system sail through load testing and transition smoothly into service? Occasionally yes. But more frequently the system starts to fail as load starts to be applied, even with small increases in volume.. For the first time there are concurrent demands on the system and arbitration over resources is required. Paths through code that have never been executed are triggered, situations arise that nobody really thought through. Transactions fail. Systems crash. After these problems are fixed and more load is applied in a test, we then encounter problems like resource exhaustion, buffer overflows, timeouts and inconsistent behaviour. The real work needed to turn a functional pre-production system into a robust solution has only just begun.

Examples abound of products that failed when load testing started and, after lots of effort, stress and expenditure, have been shelved. Worse still are the ones that missed load testing altogether and failed dramatically during live operation. An internet portal developer recently stopped development of a new service, one that had completed functional development, when load testing revealed fundamental structural problems and inefficient coding which led to a poorly performing and unstable system.

So what should you do to avoid these risks? We all know it is better to find faults early when they cost far less to fix yet load testing is still left until as late as possible. The types of faults it finds frequently need architectural changes and major rewrites which are by then are hugely expensive to implement. The answer is that you should start early. Different forms of load testing should be repeatedly applied throughout the project to identify problems early and to check that the system is not going off track.

This is a natural extension of the practice of test led development. Test led development, where automated tests are written first and code must pass these tests as it is developed, offers major benefits. However, in its current form, the focus of this testing is on functionality. As it evolves the functional status of the software is always known and hence manageable, functional faults are nipped in the bud avoiding high cost fixes, the functional risk is greatly reduced. Not so other risks. If a project performs early and continuous load testing it gets much wider and comprehensive risk reduction. To make this effective:

  1. Study the system and perform a risk analysis to help to order the threats to the system, this will help you to prioritise load testing activities.
  2. Collect data to allow comparison of the efficiency of different builds. This permits monitoring of the long term trend, ?Is the system using more and more processor time to do the same work?? This data can be used to predict resource requirements at different levels of demand and so support scalability predictions.
  3. Execute tests that aim to assess the behaviour of the system and to trigger faults under load. Use workloads that simulate expected patterns of demand to observe the aggregate behaviour of the system. Use specially targeted extreme workloads to probe the vulnerabilities of the system.
  4. Include the full spectrum of load tests into the test suite. This means performance testing with typical and busy period work loads; stress testing to check both atypical demand spikes and resource exhaustion impacts; endurance testing that uses both operational period and cumulative operation tests; reliability testing that runs lots of transactions and then checks whether occasional transactions fail; concurrency testing of two users working on the same account at the same time.
  5. Design measurement activities as scientists would design an experiment, design them to provide data that can be analysed. Sample the system under different steady state workloads to provide multiple data sets to support interpolation. Chose the workloads to permit estimation of the resource costs for each transaction type.
  6. Target the middleware first with generic activities and evolve the suite as functionality is developed. Start early and then test each incremental release of the system, firstly with the previous suite and then with a modified suite that addresses new functionality.
  7. Invest the time and resources to work at a representative scale. Maybe the test bed can?t be full scale but it should not be two orders of magnitude smaller than the intended system. Be smart and innovative to use resources effectively to provide an appropriate scale test bed. The costs that will be incurred if this is not done will far exceed the cost of providing the test bed.
  8. Don?t delay; test an increment as soon as possible. Don?t skip one or you?ll end up skipping them all. Compare the measurements and behaviour with the previous one, is it better or worse?
  9. Provide a background load for functional testing. Features that work offload may fail when the system has other things to think about.
  10. Consider occasional events such as server failures and reconfiguration of the system. Do these need to be tested under load?

In conclusion, you need to incorporate load testing throughout the development process. Leaving load testing until the final run in to live service is a recipe for disaster. If this became common practice then a lot more applications and systems that work would be delivered on time and to budget.

? Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. Popular services include performance testing and business continuity testing.

Acutest Ltd
Blackwell House
Guildhall Yard
London EC2V 5AE 
Website: http://www.acutest.co.uk/

Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

Spanish operator Telefonica today announced that it has agreed to make a cash offer to purchase UK-based mobile operator O2 for approximately ?17.7 billion (US$31.5 billion). Telefonica said the move would allow it to provide enhanced scale by entering Germany and the UK with a critical mass, and would...
Link

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

People often equate load testing with performance testing. Load testing is seen as a way of answering the question ?How fast does the system respond?? This view then tends to mean that load testing is seen as an end of project activity. Only at the end of development will we have the final implementation for performance testing and so we can confirm only then that it performs quickly enough in the

This post originally appeared on an external website

by rooymans

Spanish operator Telefonica Moviles today reported consolidated net income for the quarter to end-September increased 24% year on year to reach Eur614.9 million (US$738 million), while revenues increased 45.9% year on year to Eur4.3 billion. Operating income at the operators home market remained flat year on year at Eur1.1 billion,...

Link

This post originally appeared on an external website

by rooymans

Dutch telco KPN announced today that total operating revenues from its mobile operations stood at Eur1.44 billion (US$1.6 billion) for the three months to end-September, compared with Eur1.35 billion last year, while total operating profit for the mobile business dropped to Eur196 million in 3Q05 from Eur221 million in 3Q04....

Link

This post originally appeared on an external website

by rooymans

Microsoft Chairman Bill Gates sent a memo last month alerting his executives to the dramatic changes facing the software business. Two days earlier, Chief Technical Officer Ray Ozzie, who is leading the services push, outlined the challenges as he saw them in his own memo. In it, Ozzie asked, How can we utilize our extant products and our knowledge of the broad historical adoption of forms-based

This post originally appeared on an external website

by rooymans

UK MVNO Virgin Mobile today announced total revenues rose 7% year on year to ?274.6 million (US$471.7 million) for the six months to end-September, while service revenues were also up 9% year on year to ?250.8 million. Operating profit at the MVNO rocketed 27% year on year to ?45.5 million...
Link

This post originally appeared on an external website

by rooymans

LogicaCMG announced today it has won a contract to provide its uOne next generation voicemail and videomail solution to Portuguese operator Optimus. The service will run from the two major cities in Portugal, Porto and Lisbon. LogicaCMG's uOne solution will also allow Optimus to support their legacy voicemail solution, enabling...
Link

This post originally appeared on an external website

by rooymans

Finnish vendor Nokia announced today it has signed a definitive agreement to acquire mobile application and messaging provider Intellisync. Under the terms of the agreement Intellisync stockholders will receive US$5.25 per share in cash for each Intellisync common share, giving the company an enterprise value of around US$430 million. The...
Link

This post originally appeared on an external website

by rooymans

Testing cannot ensure complete eradication of errors. Various types of Testing have their own limitations. Consider Exhaustive black box and white box testing, which are practically not "Exhaustive" to the verbose as they are required to be, owing to the resource factors. While testing a Software Product, one of the most important things is the design of effective test cases. A Tester tries to

This post originally appeared on an external website

by rooymans

Agile methodologies are known to work fine for small projects, if approached correctly, still there is some argues about their scalability to large projects. I believe that the root cause of the issues is not the size of the project. It is the business size/goal. Contributing issue is technology complexity. Those are my assumptions ? mostly theoretic and only partly based on experience. More over

This post originally appeared on an external website

by ricki henry

Anything that works for small projects should work for large projects since one of your first objectives in a large project is to break it down into smaller deliverables, i.e. smaller projects.  The failure of any methodology with a large project is rarely the methodology but more frequently the analysis-paralysis that you see with large projects.  This means that no matter what methodology you use, it will "not work".  The major obstacle in breaking larger projects down is the argument that "if we don't know all of the requirements up front, the databases we design may not work causing major redo costs", and "if we break it down we'll be doing duplicate testing which will cost more and take more time". Well folks, break it down into smaller deliverable and do your CBA for each deliverable!.  Assume there will be "redo costs", yes, there will be retesting costs.  So if the business benefits of early deployment outweigh the costs proceed, if not, don't and combine deliverables until you have the smallest amount possible that will bring the benefit to justify the costs.  In smaller enhancement projects we don't bemoan the fact that if we had know about the current project with the last enhancement we could have saved development costs, testing costs and reduced costs of adding new data, do we? 

ricki henry (rickihenry@hotmail.com) sr. project manager with 15 years experience

by jslarochelle

The idea that you can break a large project into small iterations that can be managed the same way as the iterations in a small project is too simplistic. For one thing, in a truly large project you will get personnel turnover that is not present in small project and that requires better documentation ( requirements, design) than in a small project.

by a contractor

What is wrong with better documentation?  If the large project had come through as a series of smaller requests, the same problems would still be there.  Ricki is on the moeny and jslarochelle is a classic IT monolithic maniframe thinker.  In today's world you have to be able to not only provide value very quickly, but be able to bring a large project down to a minimal staffing level or close it down because business resources are needed elsewhere.  PM that can live in that world are working and making money.  Classic IT, monolithic, maniframe thinking PM and consultants are on the street or working in environments that can not continue to support that way of doing business.  Learn to be Agile or start looking for another line of work.

by jslarochelle

A contractor is certainly right when he says "learn to be Agile" and I do embrace Agile methodologies. However, I think the most successfull people will adapt and know when to move away from Agile into more formal methods when needed. Fortunatly most of us will rarely have to move very far away from the Agile side of the spectrum because it is appropriate for a large set of projects. I still think that those that stick fanatically to Agile methods even when not appropriate will suffer needlessly.

I think the more you know the better you are at surviving in the software business and that includes Agile methodologies as well as more formal and heaviweight methods. And again moving away from Agile and using more formal methods doesn't mean moving to the other extreme.

 

 

by rooymans

Swiss operator Swisscom announced today its Solutions unit will acquire some of Siemens Switzerland's converged solutions business for SFr25 million (US$3.2 million). As part of the deal, Swisscom Solutions will take over the support and care of medium and large business customers from Siemens Switzerland; and 120 service and sales...

Link

This post originally appeared on an external website

by rooymans

German vendor Siemens today said it has been contracted by Indonesian operator Hutchison Telecom Indonesia to build, implement, operate and maintain its forthcoming GSM/WCDMA network. Hutchison Whampoa struck an agreement in March to acquire a 60% equity stake in 3G licensee Cyber Access Communications for a cash consideration of...

Link

This post originally appeared on an external website

by rooymans

Wie das Nachrichtenmagazin "Profil" in seiner aktuewllen Ausgabe berichtet, wird tele.ring-Chef Michael Krammer zum deutschen Mobilnetzbetreiber E-Plus wechseln. Der Vertrag soll vor wenigen Tagen unterzeichnet worden sein. Krammer wird dort die Nachfolge des mit Jahresende ausgeschiedenen Vorstandschefs Uwe Bergmann ?bernehmen.
?bernahme

Als Datum f?r seinen Amtsantritt wurde der 1. M?rz

This post originally appeared on an external website

This discussion is closed to new replies.