Skip to content

TechRepublic

  • Top Products Lists
  • AI
  • Developer
  • Payroll
  • Security
  • Project Management
  • TechRepublic Academy
  • Innovation
  • Cheat Sheets
  • Big Data
  • Tech Jobs
  • TechRepublic Premium
  • Top Products Lists
  • AI
  • Developer
  • Payroll
  • Security
  • Project Management
  • TechRepublic Academy
  • Innovation
  • Cheat Sheets
  • Big Data
  • Tech Jobs
  • See All Topics
  • Sponsored
  • Newsletters
  • Forums
  • Resource Library
TechRepublic Premium
Join / Sign In
CXO

Images: Testing Microsoft’s Experimental Platform

By Andy Smith April 16, 2008, 5:11 AM PDT

Image
1
of 7

Images: Testing Microsoft’s Experimental Platform

Hippos

Hippos

The Microsoft Experimentation Platform wants to take decision making from the Hippo (highest paid person’s opinion) and take the guesswork out of project planning. The goal of General Manager Ronny Kohavi is to build a system that would map user activities using machine-learning algorithms.rnRather than make changes on the run, ask your customers first.rn

Read Mary Jo Foley’s interview with Ronny Kohavi.

Images: Testing Microsoft’s Experimental Platform

Measuring the success of a concept should be simple according to this diagram. Take two versions to test, a control–usually the live version, and the treatment which is usually the new idea. Then you simply collect the data and analyze. rnrn

Following is from a white paper from Microsoft Experimentation Platform: Practical Guide to controlled experiments on the Web: Listen to your customers not the Hippo.

Images: Testing Microsoft’s Experimental Platform

Ads--yes or no

Ads--yes or no

Ads were palced at the bottom of this shopping section to determine their worth. Clickthrough rate was 0.49 percent lower and page views were down 0.35 percent on the page with extra ads. Apparently not significant but the expected revenue was less than the loss of revenue from lost clicks, so the project was scrapped. rnrn

Click on the image to enlarge.

Images: Testing Microsoft’s Experimental Platform

Solitaire or Poker?

Solitaire or Poker?

This experiment ran in Windows Marketplace/Game Downloads. The Solitaire game had a higher clickthrough–61 percent preferred Solitaire.

Images: Testing Microsoft’s Experimental Platform

Soft vs. Contrast

Soft vs. Contrast

High-contrast text was tested against softer colors. The high-contrast page drew 0.9 percent more queries per user and 3.1 percent more ad clicks.

Images: Testing Microsoft’s Experimental Platform

Feedback A vs. Feeback B

Feedback A vs. Feeback B

Feeback A puts everything together while Feedback B is a two-step process–a question following the ratings. Feedback B got more than double the response rate.

Images: Testing Microsoft’s Experimental Platform

Feedback C

Feedback C

This feedback is also a two-stage process. It outperformed Feedback B by 3.5 times.

  • CXO
  • Account Information

    Share with Your Friends

    Images: Testing Microsoft’s Experimental Platform

    Your email has been sent

Share: Images: Testing Microsoft's Experimental Platform
Image of Andy Smith
By Andy Smith
  • Account Information

    Contact Andy Smith

    Your message has been sent

  • |
  • See all of Andy's content

TechRepublic Premium

  • TechRepublic Premium

    Best of Confluent Current 2023: The State of Data Streaming

    TechRepublic Premium was at Confluent’s Current 2023 event, held in San Jose, California, September 26-27. Our coverage of the event comprises an analysis of data streaming, interviews, the role of stream governance and a look at Apache Flink. From the download: Confluent used the Current 2023 ‘next generation of Kafka Summit’ event in San Jose ...

    Downloads
    Published:  September 27, 2023, 4:00 PM EDT Modified:  September 28, 2023, 1:00 PM EDT Read More See more TechRepublic Premium articles
  • TechRepublic Premium

    Quick Glossary: Fintech

    Fintech is a fast changing landscape that constantly introduces cutting-edge ideas and developments. TechRepublic Premium presents this quick glossary of fintech terms and concepts to help you understand technological breakthroughs and make educated decisions. From the glossary: DECENTRALIZED FINANCE Often referred to as DeFi, this utilizes emerging technology to remove third parties and central financial ...

    Downloads
    Published:  September 27, 2023, 4:00 PM EDT Modified:  September 28, 2023, 8:52 AM EDT Read More See more TechRepublic Premium articles
  • TechRepublic Premium

    Anti-Money Laundering Policy

    Money laundering poses a detrimental impact on businesses and the economy as a whole. According to the United Nations Office on Drugs and Crimes, the amount of money laundered around the world in a single year is presumed to be 2–5% of global GDP, which is roughly $800 billion to $2 trillion. The purpose of ...

    Downloads
    Published:  September 27, 2023, 4:00 PM EDT Modified:  September 28, 2023, 9:14 AM EDT Read More See more TechRepublic Premium articles
  • TechRepublic Premium

    Hiring Kit: Video Game Audio Engineer

    Modern video games rely on a cornucopia of sounds to engage and engross players. Whether it is ambient sound to set the mood, music to invoke a feeling or dialog to tell the story, sound is vital to the immersive experience of a video game. The individual responsible for meshing all those disparate sounds together ...

    Published:  September 27, 2023, 4:00 PM EDT Modified:  September 28, 2023, 10:00 AM EDT Read More See more TechRepublic Premium articles

Services

  • About Us
  • Newsletters
  • RSS Feeds
  • Site Map
  • Site Help & Feedback
  • FAQ
  • Advertise
  • Do Not Sell My Information
  • Careers

Explore

  • Downloads
  • TechRepublic Forums
  • Meet the Team
  • TechRepublic Academy
  • TechRepublic Premium
  • Resource Library
  • Photos
  • Videos
  • TechRepublic
  • TechRepublic on Twitter
  • TechRepublic on Facebook
  • TechRepublic on LinkedIn
  • TechRepublic on Flipboard
© 2023 TechnologyAdvice. All rights reserved.
  • Privacy Policy
  • Terms of Use
  • Property of TechnologyAdvice