14 replies
by mckaytech
A couple of hackneyed phrases come to mind:
1. there are lies, damned lies and statistics
2. you get what you pay for
The danger in vendor-funded tests is not that they won't be conducted rigorously but simply that bad results will never see the light of day. As an indication of the expected theoretical peak performance of the product, the numbers probably still have some validity with regarded with the usual qualifications about comparing apples to apples.
However, I've been involved with I.S. for a long, long time and I have yet to be involved in a critical product purchase that was more about performance than features. A system that is capable of blinding speed performing transactions you don't want to perform has less value than one that does what you want at some lesser speed.
paul
by simonc
1. No.
2. Little to very little.
3. Not much at all.
Hi!
I agree with the above and as a commment I can say, if Exchange performed so well how come they couldn't run exchange on hotmail.
They had to use Sun Sparc high end servers and Soalris with unix mail server in order to keep hotmail runningsince exchange couldn't handle the load.
Not sure Notes would be able to handle it either.... but.....
/Hasse
by arwjnplanotx
The only information I expect to gain from any review is point references. When I read a review I'm looking for what the product does and what it doesn't do. I have had my fingers burnt too many times to believe even an unbiased review. I'm talking about reviewers that are not influienced by the vendors.
If, however, a reviewer makes a point that a software or hardware product stands out above the rest I will take a closer look at that product. I will not accept his/her review as gospel under any circumstances.
My favorite way to make an initial evaluation of a product is to consult associates who have used or are familure with the product. My second way is to obtain an evaluation copy and set up a test project.
By no means would I plan an implementation of a product on reviews alone.
As far as reviews go, I believe most vendors would want an unbiased review so their customers don't find out later that a product doesn't support a needed function. If I were a manufacturer Iwould not want to ha
by nishpc
The Gartner article was a pretty fair analysis of the issues with any comparative test or any scientific experiment for that matter. They have also included some data that indicates that Msft has systematically overstated the capabilities of Exchange. I ahve to agree with that. We gave up on Exchange's ability to efficiently synchronize data with travellers and tried other solutions after buying Exchange for that very purpose based somewhat on vendor claims.
Vendor-sponsored testing shouldnot be relied on for critical decision points, and I do not rely on them.
An issue worth thinking about may be that the lab conducting the test should have the integrity to provide adequate information to place the test results in the proper perspective. Why should consumers trust future reports of ZDLabs testing in their magazines or other venues when they are party to misleading results/reports like these...?
by mcorr
Microsoft doesn't overstate capabilities, they "creatively" state the abilities. That means they state the great things, but not the bad. Vendor tests are virtually useless, mostly useful so execs can state "we have the best x" and back it up withtheir rigged test.
Good luck getting your impartial test off the ground. Microsoft will not participate in any test they won't win, because they will run the test internally first.
How about this for a fair test:
1. Spinning off a tiny company for just the test.
2. Hire a consulting company that specializes in setting up each to configure a best possible system (within a budget)
3. Publish the numbers, and let MS revoke the licenses from the small spin off company.
by smithjj
Do you think that Microsoft tends to overstate the capabilities of
Exchange Server?
I'm sure what they state is accurate and true. However, what they
state, as the article avers, may not be relevant or useful, and may
be misleading. Comparing software with different hardware configurations
is meaningless. I'm surprised MS even bothers, for it undermines any
credibility they have. Only the guillable will be taken in by this test.
How much weight should vendor-sponsored comparisontests have
when it comes to influencing decisions on software and
hardware purchases for an organization?
If the vendor tests exactly what the company needs, then it's useful.
Don't bet on it.
How much do you depend on these comparison tests when
making a decision?
None whatsoever. We do our own tests, and we choose hardware and
software more on functionality than performance.
This discussion is closed to new replies.
Trending Windows Forum posts
Sort by Trending