Networks

Ask something
Discussion Locked

RHL vs. ws2k3: too funny for words

0 helpful
32 replies

by apotheon

Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

Imagine my surprise. I was browsing through an article comparing benchmarks for Athlon, P4, and Pentium M processors, and I saw an ad claiming that Windows Server 2003 "performs" better than comparable Red Hat Linux servers. I thought about following the link from the ad to Microsoft's "get the facts" site, where MS posts all of its bribed FUD-supporting studies, to see more about this. For one thing, I was highly suspicious of the use of the term "Red Hat Linux" in the ad. It turns out I was right to be suspicious.

As you can see if you go to http://tinyurl.com/4vvjc and download the PDF there, Microsoft Windows Server 2003 was compared, in several configurations, to Red Hat Linux in several configurations.

I don't pretend to know the first thing about Red Hat Linux Advanced Server, except that it's not primarily optimized for x86 architectures and it's not meant to be wasted as a webserver. It's a datacenter OS, fercryinoutloud. I do, however, know that the other Red Hat OS used, Red Hat Linux 8.0, is [b]way[/b] too old to be meaningfully compared to Windows Server 2003 for performance purposes.

I also notice that out of four major benchmark comparisons highlighted on the first page of the report, three are comparisons between IIS and Apache, and one is a bit more complicated. The "more complicated" comparison involves pitting IIS against both Apache and TUX webservers on Advanced Server, but only Apache on RHL 8.0. Interestingly enough, it was TUX on RHL that just stole IIS's lunch money, beat it up, made it cry uncle, and spat on it for good measure on Windows 2000 Server in the year 2000.

So. Let's get this straightened out:

1. Microsoft is reporting a win for Windows Server 2003 with IIS over Red Hat offerings with Apache, in terms of performance.

2. It chooses two different webservers and two different Linux versions from Red Hat as the competitors against which IIS will compete.

3. The two webservers against which it will compete are Apache, known for flexibility, power, stability, and security, and TUX, primarily known for beating the pants off IIS. Well, if it beats out Apache, that's kind of a bummer for Linux fans, but life goes on. If it beats out TUX in a fair fight, there's definitely something startling to see.

4. The two OSes against which Windows Server 2003 competed include Red Hat Linux 8.0, which has been superseded by several 8.x versions, several 9.x versions, several 10.x versions, Fedora Core 1, Fedora Core 2, and Fedora Core 3 (or, if you prefer, relevant versions of Red Hat Enterprise Linux instead of Fedora Core). Obviously, if this is the only OS against which ws2k3 is competing, Microsoft is essentially cheating. That's like comparing Windows XP Professional against Windows 3.11 on DOS. At least there's another Red Hat OS being offered.

5. The other Red Hat version used is Advanced Server 2.1. Well, now we're getting somewhere, right? It's a modern Red Hat OS, relatively speaking. It only came out something like one or two years before ws2k3. Well, I guess we can't expect something actually contemporary from a comparison in which Windows wins, but at least we're getting something close. Unfortunately, as I already noted, AS is not designed for use as an x86 webserver. Okay, so it's a little old, and it's the wrong tool for the job. I guess this doesn't count for much, after all.

6. But wait! Windows Server 2003 with IIS beat out TUX! That's gotta count for something! Yeah! Unfortunately, it was only tested against TUX on AS with Xeon processors. The strength of the TUX webserver is in its ability to work closely with standard x86 processors at the low end, like P3s (which, coincidentally, is what was running on the RHL 8.0 systems, while AS and ws2k3 ran multiple Xeons). Something tells me that TUX on RHL 8.0 might still make IIS on ws2k3 scream like a b*ch.

In the end, the whole effort really only proves one thing: Every single benchmark I've ever seen that favored a Windows platform system over a Linux system was a Microsoft-funded and Microsoft-controlled study, only incompletely reported with frighteningly good results for Linux being ignored so that only its least impressive features are compared, or a comparison of apples and oranges (like this one). In many cases, some combination of two of these, or of all three, was in effect.

This time around, the test was apparently specifically tailored to provide the best possible results for Windows systems, by setting up apples-and-oranges competitions between Windows and Linux platform webservers. This is not quite as blantant, but is at least as egregious, as Microsoft's "total cost of ownership" reported coup over Linux when it compared Windows x86 servers against Linux [b]mainframe[/b] servers!

Yeah. Folks, if Windows can't look good in a fair competition, you shouldn't be trying to deceive us with cheats. Seriously.

32 replies

by secure_lockdown9

a lot depends on when they did their study and whether Red Hat's newer Enterprise Linux AS/ES were out when they were doing their study.

I guess RHEL ver. 3 would be in the running up against Win2k3. thought i still think you are not comparing apples to apples when comparing the two.

also, naturally, if this is appearing on M$'s web site - they are not going to post anything that will hurt their product. so a slant towards M$ is jjust business as usual.

SL

by apotheon

The proper comparison would have been ES vs. ws2k3, with both Apache and TUX vs. IIS, on not only multi-processor Xeon systems but also (more) common x86 servers in both single-processor and multiprocessor configurations. Truthfully, nobody's going to host a website on eight-processor Xeon systems unless it's dealing with very high loads, and Apache absolutely kills IIS in sustained high-load scalability.

Meanwhile, TUX is optimized for (non-Xeon) standard x86 processor architectures, designed for high performance at low loads. Also, ES is designed more for the low-load services (which is what this test is really catered to) than AS. I'm sure that Apache on ES with P4 would more favorably compare with IIS with P4, and that TUX on ES with P4 would make a mockery of IIS on P4 just as Red Hat Linux 8 with TUX did in comparison with Win2k with IIS in the year 2000.

I'm not sure whether RHEL 3.0 was available at the time this benchmarking was performed, and as such I didn't get too in-depth about the failure to use something newer than 2.1, but there is certainly a problem with choosing AS 2.1 and RHL 8.0. I mean, really, there was the entire 8.x line, 9.x line, 10.x line, and RHEL 1.x line, before the 2.x ever even came out, and yet somehow this benchmark seems to expect someone to take the comparison of RHL 8.0 with ws2k3. That's just utterly absurd.

by jaqui

well, since everyone knows that windows is a virus, as well as being a virus magnet.
if miscrosoft did honest comparisons, they would go out of business.

by apotheon

That's so, unless Microsoft were to suddenly spend a lot more of that enormous marketing/FUD budget on developing value for the customer. Fat chance o' that, though.

by dafe2

......I'll have to go scrub that ol' disk of mine. Can't have that pesky data being organized by some virus.

yikes.......

by apotheon

Little did you know.

Hey, I got the Unix email virus not too long ago. The email said the following:
---
YOU HAVE NOW RECEIVED THE UNIX VIRUS

This virus works on the honor system:

If you're running a variant of unix or linux, please forward this message to everyone you know and delete a bunch of your files at random.
---

by dafe2

Well now, see, at least the penguin has honor & class. Is it too much to ask for a simple e-mail warning?

So, in Microsofts case it seems they just produced a big ol virus and sold it to a kagillion people without warning. I can't believe we're just finding this out. How rude. In all fairness, I may have skipped that chapter during some of my reads though. Can't believe I missed the magnet part too. gosh dern it.

by jaqui

you mean you never installed win95 with bios antivirus running?
it wouldn't allow windows to be installed, as it's a virus.

by dafe2

But you had to go back 9 years.

See...It wasn't so hard to give the reason for your statement was it? I'm still laughing.
Usually, people say they don't like MS & don't back it up with any intelligent or coherent response. (Or, in your case good humour & twisted reasoning) :-)

Now, in fairness, that was only on cheap system boards, that for some reason, wouldn't allow boot disc creation. Gotta hate a boot sector virus. On the other hand, it still happens today & on main stream stuff too. Hmmm... Man oh man....

Thank you Jaqui

by house

Benchmark tests are usually designed in a manipulative way in order to promote a solid advertising campaign. Being married to Intel has certainly enhanced Microsoft's ability to provide peek performance ratings in benchmark tests. The only tests that I trust are in independent, third party, and non-influenced organizations. We see the same issues in Video Cards. Some hardware is optimized by some software. This goes back to the initial development in regards to the products used, technologies exploited, and so on. The resulting mess fails miserably in compatibility standards.

They forgot to mention

1) The amount of time that it takes to tweek the security when you are dealing with IIS.

2) The ease of implimentation when using default Apache packages that are geared towards different functions. (also, the unbelievable amount of options when chosing the appropriate package)

3) Like apotheon said, testing on different hardware platforms is always neglected by MS.

4) What sort of RHL installation are they talking about? Is it the complete package? What mode are they going to run in? 4 GB of useless tools will definitely hamper performance ratings.

5) Who uses a fat GUI in server operations anyways?

* The only advantage that MS has is in the fallout of it's popularity. ie; Support, development, comfort and knowledge of the OS structure, etc.

by dafe2

An implementor of Microsoft Business solutions who thinks a baseline involves the Red Sox.

by house

...that if Microsoft had not been so dumb, hardware development might not be quite as advanced as it is right now. It sounds funny, but their negligence regarding the useless "meat" that they include, has most definitely lit a fire under the @ss of the hardware manufacturers.

Thanks to MS... for it's indirect contribution in the technological advancement of our computing systems. :)

by dafe2

Is this not typical Microsoft strategy though?

Forgive my ignorance of Linux product(s)...but as I see it from a business standpoint, Linux has gathered much support in the past few years. Microsoft is feeling some ?warmth? on it?s heels.

Most hardware firewalls today are based on Linux, many (network) appliances have embedded systems based on Linux. Most Allen-Bradley PLC's are based on some form of Linux ES.

Even Hewlett Packard Server offerings (And Others) now have Linux based offerings, and they seem to be popular.

Seems like Microsoft feels threatened to me...
I'd place no value in "published" benchmarks. impartial or otherwise. As well, FUD works both ways & is used extensively by both camps.

In fact from what's presented here, it's nothing more than good marketing.

Most "benchmarks" leave a lot to interpretation. On a lower scale, people continue to benchmark PC's and even servers. The results leave a lot more questions than answers. They all offer the same BS today.

Good business. Yeah. Unethical or deceptive?Guess it depends who you ask as an interpreter.

by house

All is fair in business. It is up to the professional to weigh and interpret the results as fact or fiction.

by aceskaraoke

I had seen the pop-up about Win Server 2003 beating out RedHat and I was wondering if Windows had really come up with a superior product for server applications.

I guess it makes sense now finding out that the comparisons were made against an old version of RedHat. It's disappointing they'd have to be that deceptive, not surprising, just disappointing.

I wonder if they ran these batteries of tests against Fedora first, didn't like the results... then ran against RedHat 10, then RedHat 9, and then finally RH8 where they found numbers they liked.

I guess they're just lucky they were able to beat a version of RedHat only a few years old and didn't have to benchmark against a liveCD distro to run the Linux Server before they could win.

by apotheon

As far as I'm concerned, outright lies and intentionally misleading marketing constitute fraud (from an ethical, though not necessarily legal, perspective; the law isn't always ethical). Fraud isn't anything I'd ever term "good business".

I agree with the "let the buyer beware" admonishment, suggesting that we should be aware of what we're buying, and should research our options. I agree with that, insofar as it's just a good thing to think for oneself, and to protect oneself from unethical people. That justifiable, wise wariness doesn't give anyone ethical license to coerce through deception, though.

by dafe2

I think we'd both agree, this is a great example of just how fine the line is - between fraud & misrepresentation, it's also a bigger commentary on where we're at today in this or any business.

Ethics asside, Microsoft used an age old legal business strategy. Benchmarking BS & misrepresentation has been arround since IBM invented FUD after all.

This happens in hardware 'wars' too. As you say, legal is not necessarily ethical or even justice.

We sure as hell don't like this kind of stuff, but marketers & sales people eat this stuff up and to them it's "good business". (Note the question mark on my post)

Absolutely, let the buyer beware. Hire the people that understand the technical side of IT implementations.

Great post.

by house

Political business tactics are great. Deceiving honest folks for profit rules. I love ripping people off. Just kidding. I do respect business for what it is though. :)

The same thing applies to a neutral paper that is written in such a way that it excludes arguments for the other side of the spectrum, without actually smelling like a biased rag. This kind of stuff is everywhere.

"And if all others accepted the lie which the Party imposed ? if all records told the same tale ? then the lie passed into history and became truth. 'Who controls the past' ran the Party slogan, 'controls the future: who controls the present controls the past." Orwell, George [i]1984[/i]

by jaqui

I have an old copy of mandrake. ( 6.1 "Helios" )
we could give it to m$ to use for testing purposes.

but it would probably still kick the but out of iis.
if the box was set up for a server not a workstation.

by aceskaraoke

Misrepresentation and fraud are basically the same. Seems to just be a way to differentiate whether or not one gets sued/arrested or just vehemently complained about.

It's hard to learn about ethics when all around you have lost or sold theirs.

by dafe2

The difference between fraud and misrepresentation is simple - not all misrepresentations are fraud, but all fraud involves a misrepresentation. Fraud is intentional misrepresentation. Fraud requires an element of intent.

"A lie which is half a truth is ever the blackest of lies, . . . a lie which is all a lie may be met and fought with outright, but a lie which is part a truth is a harder matter to fight."

by kaceyr

And this is coming from a developer who, as you correctly interpreted in another discussion apotheon, has his income tightly coupled with the Microsoft product lines.

I haven't seen a benchmark, of any kind, since the early 80's that was honest. The manufacturers carefully select the competition, pick the factors to be benchmarked, then get to approve the results (or change the rules and re-test) before publication.

From my perspective, it's all a bunch of crap. I don't know who ran the benchmarks, but I'm certain they posted large profits from Microsoft.

The only way that I can think of to get a fair benchmark is for an independent company, person, or organization to run the benchmarks themselves, selecting only the latest stable release from each of the competitors, then running each on the same machine. Not a similar machine, but the EXACT SAME MACHINE! Otherwise, no matter how careful you are, your benchmarks are compromised. Of course this also means that the product selection has to be able to run on the same hardware, but isn't that the point?

In my opinion, the single most important factor of the testing is that the tester has neither interest, nor care, over who wins. Research and analysis without bias. Hard to come by, but I know there's some out there.

Since I don't have Linux installed, I can't run my own benchmarks. I can tell you that the (slightly flawed) install that I have of Linux running under a Microsoft Virtual Machine on Windows XP Pro, runs with Apache and serves up pages faster than the IIS 5 running on the XP host. This does not speak well of IIS, and since both the Apache and the IIS are a bit dated, is seems to be a more honest comparison than the Microsoft sanctioned test.

On a side-note, for the past few years I've been using web based (HTML and DHTML) presentations at meetings running from a Microsoft Windows laptop (talk about easy to write presentations). IIS is just too bulky for a presentation, so I use a web server called Xitami. I have it running on a USB dongle, complete with the presentation(s). Very effective, very low cost.

Xitami is, in fact, free. Yup, Open Source. At the risk of sounding like an ad, if you're interested check out the company on the web, iMatix Corporation (www.imatix.com).

by apotheon

It's good to hear of other options for people wedded to the MS Windows platform than the actual MS product line. I'll look into Xitami. Thanks for the mention.

by drei22

Another interesting little fact that I am seeing is that upper management in my chain of command is asking more and more questions about linux based operating systems in general. Microsoft definitely pays my bills. But slowly and surely I think I may have to include Red Hat as a personal cash cow as well.

by house

Microsoft is perhaps the Ministry of Truth.

I actually did an independent study in high school regarding "politically correct" and "new-speak". This seems to have been my main interest during my younger years considering I had also chosen Utopian societies as my next research project.

PS - This one is for your telescreen....

http://tinyurl.com/4ggbz

[i]creepy[/i]

:)

by dafe2

Don't forget to include the Lazy Programmers out there that have Moores Law framed & mated on his wall beside their monitors?

by dafe2

To remember what hubris meant. That's the other IE security issue.
A programmer with an overbearing presumption that a competent user is sitting on the other end of the interface he's so proud of.

I'd settle for the ones that just take pride.........but I think a good chunk of them are unemployed a the moment.

by house

I'm glad that I'm not the only one who went to the good ol' dictionary.

[i]hubris[/i] overbearing pride or presumption; arrogance; Gates-ism

:)

by jaqui

~l~
it was the heads up to me to look for alternatives.
been using linux only for years now because of it.

This discussion is closed to new replies.