It appears that VMware is leading the effort to offer a new benchmark that will measure the performance of virtual machines. In fact, it has just unveiled the industry’s first benchmark designed to measure the performance of enterprise workloads on virtualized environments.
Currently considered both to be the pioneer as well as the current market leader in x86 virtualization, it really should come as no surprise.
According to Andrea Eubanks, a senior director for enterprise and technical marketing at VMware, the problem with traditional benchmarks is that they were designed to test how one workload performs on a single server.
These traditional tools become totally useless when confronted with a production virtualized environment. In such a scenario, there will typically be at least several applications running on the same physical piece of hardware.
Hence what VMware did is to use several existing benchmarks that allow for running multiple, heterogeneous workloads in parallel to provide for a score to measure the system performance. In effect, this allows users to compare the performance and scalability of different virtual environments and platforms.
The VMmark benchmark scales by running multiple, so-called "tiles" until the host machine reaches saturation. Each of these tiles is a set of six workloads, including a data base server, mail server, Web server, Java transaction server, and a standby server, for failover or quick deployments.
The final score is based on the performance of the workloads at a given number of tiles. The two main benefits … is that the VMmark benchmark can show the right number of VMs a system can support, and it can give a proper measurement of the VMs density of a physical server.
In fact, Dell and SUn Microsystems have already published results based on the beta version of the benchmark. You can read more from the VMware press release on VMmark.