Skip to content

TechRepublic

  • Top Products Lists
  • AI
  • Developer
  • Payroll
  • Security
  • Project Management
  • TechRepublic Academy
  • Innovation
  • Cheat Sheets
  • Big Data
  • Tech Jobs
  • TechRepublic Premium
  • Top Products Lists
  • AI
  • Developer
  • Payroll
  • Security
  • Project Management
  • TechRepublic Academy
  • Innovation
  • Cheat Sheets
  • Big Data
  • Tech Jobs
  • See All Topics
  • Sponsored
  • Newsletters
  • Forums
  • Resource Library
TechRepublic Premium
Join / Sign In
Data Centers

Intel trots out misleading benchmark charts

By mateoconner@gmail March 16, 2007, 5:34 AM PDT

Image
1
of 7

Intel trots out misleading benchmark charts

Intel's energy efficiency slide is like a comparison of apples to eggplant.

Intel's energy efficiency slide is like a comparison of apples to eggplant.

In a manipulative attempt to demonstrate leadership over AMD for the press and analysts, Intel twists benchmark data to make it look as though it’s latest Xeons are outperforming the best AMD has to offer. Yet, upon closer inspection, most of the data being shown is irrelevant.

Using line charts to present benchmark data is tricky to begin with. Usually, you have performance points going up the Y axis and along the X axis, you might have variable loads. But you’d never do what Intel has done here.

In this graph designed to show leadership in energy efficiency, not only do the benchmarks used change from one data point to the next, but so too do the chips being compared (as evidenced by the extensive footnoting). Whereas the Intel chip stays constant (a dual core Xeon 5160), the AMD chip to which it’s compared changes from one benchmark to th next. In some cases, it’s the newest chip. In others, an older generation Opeteron is used. Still others, Intel is comparing its dual core to an AMD single-core

To compound matters, for two of the benchmarks that show Intel taking a commanding lead, both of the benchmarks are six year old benchmarks for which newer versions exit. In fact, of the last ten benchmarks to the right side, all but one compare the Xeon 5160 to a chip it shouldn’t be compared to, or use a retired benchmark.

Where the little yellow question mark appears, we don’t even know what AMD chip was used there. It doesn’t say. Maybe it was an old 32-bit Athlon?

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.

Intel trots out misleading benchmark charts

Single Core chips broken out.

Single Core chips broken out.

This slide, taken from a presentation give one week after the previous slide appeared, is a slight improvement in that Intel is calling out single vs. dual core comparisons. But even so, the rest of the problems remain.

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.

Intel trots out misleading benchmark charts

More Apples vs. Eggplants, this time around performance

More Apples vs. Eggplants, this time around performance

Although we don’t have all the annotations on this slide like we did on the first, we could have simply taken them from the first slide and overlayed them onto this one. Pretty much every benchmarking felony committed on the first slide in this presentation was committed on this slide as well, thus completely calling into question Intel’s characterization of its performance as “Breakaway.” Based on this data, it’s nothing of the sort. Would Intel consider the Opteron’s advancements over its old 386 technology a sign of a breakaway?

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.

Intel trots out misleading benchmark charts

More benchmarking silliness

More benchmarking silliness

In one version of this slide, Intel cites two SAP benchmarks in declaring “leadership benchmarks” of it’s latest greatest Xeons over AMD. But the footnotes reveal that the SAP performance differences are over AMD’s older family of Opterons (the 885). In fact, they’re the 2.6Ghz 885’s which weren’t even the fastest versions of those chips. AMD had a 2.8Ghz version as well.

As if this slide doesn’t have enough problems, it too cites the 6 year old SPEC benchmarks for which newer versions have bene issued by their authors.

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.

Intel trots out misleading benchmark charts

Oops, missing footnote

Oops, missing footnote

In this version of the previous slide (the version used to show the press and analysts on February 28th), the fine print that shows how the SAP 4P benchmark was not run against the latest greatest Opteron (the 2200SE) is missing. With that footnote missing, it would be easy for the press to assume it was against the newest generation of Opteron processors. But in reality, as can be seen fro the previous slide, it was not.

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.I

Intel trots out misleading benchmark charts

Why bother with footnotes?

Why bother with footnotes?

We were shocked when we saw this benchmark chart. There were so many footnotes to denote how the chip being compared to was not an Opteron 2XXX (as the headings say) that we wondered why not do it the other way around? Instead of saying which chips were not an Opteron 2XXX, footnote which ones were. There would have been fewer footnotes!
Check out the top right hand quadrant of this chart. The heading says “vs. Opt 2XXX.” Yet, not one single system under that was an Opteron 2XXX. We were imagining the press and analysts looking at this slide, not paying attention to any of the footnotes to realize that there wasn’t a single shred of truth t the heading.

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.

Intel trots out misleading benchmark charts

Even worse than the last slide

Even worse than the last slide

In this slide, Intel even had the audacity to use big bold text that says “Continued Leadership vs. Latest Opteron.” Yet, 21 of 32 results shown don’t involve the latest Opteron or even a member of th the latest Opteron family. And again, just like with the last slide, the top right hand quadrant of data has “vs. Opt 2XXX” in the heading. But none of the chips listed below it were an Opteron 2XXX. Everyone of them had to be footnoted.

For David Berlind’s write up on Intel’s incredibly misleading usage of benchmarks, see his post in ZDNet’s TestBed blog.

  • Data Centers
  • Account Information

    Share with Your Friends

    Intel trots out misleading benchmark charts

    Your email has been sent

Share: Intel trots out misleading benchmark charts
Image of mateoconner@gmail
By mateoconner@gmail
  • Account Information

    Contact mateoconner@gmail

    Your message has been sent

  • |
  • See all of mateoconner@gmail's content

TechRepublic Premium

  • TechRepublic Premium

    Power Checklist: Managing and Troubleshooting Android Devices

    The popularity of Android devices means a strategy for management and troubleshooting is crucial for employee and company success. This checklist from TechRepublic Premium can help your organization get the most out of its investment in Android devices (or that of your employees if they are using their own). From the checklist: DEVICE TROUBLESHOOTING For ...

    Published:  October 3, 2023, 4:00 PM EDT Modified:  October 4, 2023, 9:00 AM EDT Read More See more TechRepublic Premium articles
  • TechRepublic Premium

    Hostile Workplace Prevention Policy

    A hostile work environment affects the psychological well-being of employees, which may lead to decreased productivity. Every enterprise should establish a written policy, like this one from TechRepublic Premium, and require every employee, contractor and third-party vendor to read, acknowledge and agree to comply with its principles. From the policy: FILING A COMPLAINT The company ...

    Published:  October 3, 2023, 4:00 PM EDT Modified:  October 4, 2023, 9:00 AM EDT Read More See more TechRepublic Premium articles
  • TechRepublic Premium

    Quick Glossary: Cybersecurity Attacks

    It doesn’t matter whether your organization is a huge multinational business enterprise or a one-person operation. At some point, your computer networks and systems will be attacked by someone with criminal intent. Cybersecurity attacks, in all their various forms, are inevitable and relentless. This quick glossary from TechRepublic Premium explains the terminology behind the most ...

    Published:  October 3, 2023, 4:00 PM EDT Modified:  October 4, 2023, 9:00 AM EDT Read More See more TechRepublic Premium articles
  • TechRepublic Premium

    Hardware Decommissioning Policy

    Every hardware device used by an organization has a finite lifespan or viable period of use, and the retirement of old gadgets and the acquisition of new ones is a part of life with technology. This policy from TechRepublic Premium provides guidelines for the appropriate removal and disposal of all technological hardware used by the ...

    Published:  October 3, 2023, 4:00 PM EDT Modified:  October 4, 2023, 9:00 AM EDT Read More See more TechRepublic Premium articles

Services

  • About Us
  • Newsletters
  • RSS Feeds
  • Site Map
  • Site Help & Feedback
  • FAQ
  • Advertise
  • Do Not Sell My Information
  • Careers

Explore

  • Downloads
  • TechRepublic Forums
  • Meet the Team
  • TechRepublic Academy
  • TechRepublic Premium
  • Resource Library
  • Photos
  • Videos
  • TechRepublic
  • TechRepublic on Twitter
  • TechRepublic on Facebook
  • TechRepublic on LinkedIn
  • TechRepublic on Flipboard
© 2023 TechnologyAdvice. All rights reserved.
  • Privacy Policy
  • Terms of Use
  • Property of TechnologyAdvice