Imagine my surprise. I was browsing through an article comparing benchmarks for Athlon, P4, and Pentium M processors, and I saw an ad claiming that Windows Server 2003 “performs” better than comparable Red Hat Linux servers. I thought about following the link from the ad to Microsoft’s “get the facts” site, where MS posts all of its bribed FUD-supporting studies, to see more about this. For one thing, I was highly suspicious of the use of the term “Red Hat Linux” in the ad. It turns out I was right to be suspicious.
As you can see if you go to http://tinyurl.com/4vvjc and download the PDF there, Microsoft Windows Server 2003 was compared, in several configurations, to Red Hat Linux in several configurations.
I don’t pretend to know the first thing about Red Hat Linux Advanced Server, except that it’s not primarily optimized for x86 architectures and it’s not meant to be wasted as a webserver. It’s a datacenter OS, fercryinoutloud. I do, however, know that the other Red Hat OS used, Red Hat Linux 8.0, is [b]way[/b] too old to be meaningfully compared to Windows Server 2003 for performance purposes.
I also notice that out of four major benchmark comparisons highlighted on the first page of the report, three are comparisons between IIS and Apache, and one is a bit more complicated. The “more complicated” comparison involves pitting IIS against both Apache and TUX webservers on Advanced Server, but only Apache on RHL 8.0. Interestingly enough, it was TUX on RHL that just stole IIS’s lunch money, beat it up, made it cry uncle, and spat on it for good measure on Windows 2000 Server in the year 2000.
So. Let’s get this straightened out:
1. Microsoft is reporting a win for Windows Server 2003 with IIS over Red Hat offerings with Apache, in terms of performance.
2. It chooses two different webservers and two different Linux versions from Red Hat as the competitors against which IIS will compete.
3. The two webservers against which it will compete are Apache, known for flexibility, power, stability, and security, and TUX, primarily known for beating the pants off IIS. Well, if it beats out Apache, that’s kind of a bummer for Linux fans, but life goes on. If it beats out TUX in a fair fight, there’s definitely something startling to see.
4. The two OSes against which Windows Server 2003 competed include Red Hat Linux 8.0, which has been superseded by several 8.x versions, several 9.x versions, several 10.x versions, Fedora Core 1, Fedora Core 2, and Fedora Core 3 (or, if you prefer, relevant versions of Red Hat Enterprise Linux instead of Fedora Core). Obviously, if this is the only OS against which ws2k3 is competing, Microsoft is essentially cheating. That’s like comparing Windows XP Professional against Windows 3.11 on DOS. At least there’s another Red Hat OS being offered.
5. The other Red Hat version used is Advanced Server 2.1. Well, now we’re getting somewhere, right? It’s a modern Red Hat OS, relatively speaking. It only came out something like one or two years before ws2k3. Well, I guess we can’t expect something actually contemporary from a comparison in which Windows wins, but at least we’re getting something close. Unfortunately, as I already noted, AS is not designed for use as an x86 webserver. Okay, so it’s a little old, and it’s the wrong tool for the job. I guess this doesn’t count for much, after all.
6. But wait! Windows Server 2003 with IIS beat out TUX! That’s gotta count for something! Yeah! Unfortunately, it was only tested against TUX on AS with Xeon processors. The strength of the TUX webserver is in its ability to work closely with standard x86 processors at the low end, like P3s (which, coincidentally, is what was running on the RHL 8.0 systems, while AS and ws2k3 ran multiple Xeons). Something tells me that TUX on RHL 8.0 might still make IIS on ws2k3 scream like a b*ch.
In the end, the whole effort really only proves one thing: Every single benchmark I’ve ever seen that favored a Windows platform system over a Linux system was a Microsoft-funded and Microsoft-controlled study, only incompletely reported with frighteningly good results for Linux being ignored so that only its least impressive features are compared, or a comparison of apples and oranges (like this one). In many cases, some combination of two of these, or of all three, was in effect.
This time around, the test was apparently specifically tailored to provide the best possible results for Windows systems, by setting up apples-and-oranges competitions between Windows and Linux platform webservers. This is not quite as blantant, but is at least as egregious, as Microsoft’s “total cost of ownership” reported coup over Linux when it compared Windows x86 servers against Linux [b]mainframe[/b] servers!
Yeah. Folks, if Windows can’t look good in a fair competition, you shouldn’t be trying to deceive us with cheats. Seriously.