Benchmarking in the Cloud: What It Should, Can, and Cannot Be
With the increasing adoption of Cloud Computing, the authors observe an increasing need for Cloud Benchmarks, in order to assess the performance of Cloud infrastructures and software stacks, to assist with provisioning decisions for Cloud users, and to compare Cloud offerings. They understand their paper as one of the first systematic approaches to the topic of Cloud Benchmarks. Their driving principle is that Cloud Benchmarks must consider end-to-end performance and pricing, taking into account that services are delivered over the Internet. This requirement yields new challenges for benchmarking and requires them to revisit existing benchmarking practices in order to adopt them to the Cloud.