Skip to content

TechRepublic

  • Top Products Lists
  • Developer
  • 5G
  • Security
  • Cloud
  • Artificial Intelligence
  • Tech & Work
  • Mobility
  • Big Data
  • Innovation
  • Cheat Sheets
  • TechRepublic Academy
  • CES
  • TechRepublic Premium
  • Top Products Lists
  • Developer
  • 5G
  • Security
  • Cloud
  • Artificial Intelligence
  • Tech & Work
  • Mobility
  • Big Data
  • Innovation
  • Cheat Sheets
  • TechRepublic Academy
  • CES
  • See All Topics
  • Sponsored
  • Newsletters
  • Forums
  • Resource Library
TechRepublic Premium
Join / Sign In
CXO

Images: Testing Microsoft’s Experimental Platform

By Andy Smith April 16, 2008, 5:11 AM PDT

Image
1
of 7

Images: Testing Microsoft’s Experimental Platform

Hippos

Hippos

The Microsoft Experimentation Platform wants to take decision making from the Hippo (highest paid person’s opinion) and take the guesswork out of project planning. The goal of General Manager Ronny Kohavi is to build a system that would map user activities using machine-learning algorithms.rnRather than make changes on the run, ask your customers first.rn

Read Mary Jo Foley’s interview with Ronny Kohavi.

Images: Testing Microsoft’s Experimental Platform

Measuring the success of a concept should be simple according to this diagram. Take two versions to test, a control–usually the live version, and the treatment which is usually the new idea. Then you simply collect the data and analyze. rnrn

Following is from a white paper from Microsoft Experimentation Platform: Practical Guide to controlled experiments on the Web: Listen to your customers not the Hippo.

Images: Testing Microsoft’s Experimental Platform

Ads--yes or no

Ads--yes or no

Ads were palced at the bottom of this shopping section to determine their worth. Clickthrough rate was 0.49 percent lower and page views were down 0.35 percent on the page with extra ads. Apparently not significant but the expected revenue was less than the loss of revenue from lost clicks, so the project was scrapped. rnrn

Click on the image to enlarge.

Images: Testing Microsoft’s Experimental Platform

Solitaire or Poker?

Solitaire or Poker?

This experiment ran in Windows Marketplace/Game Downloads. The Solitaire game had a higher clickthrough–61 percent preferred Solitaire.

Images: Testing Microsoft’s Experimental Platform

Soft vs. Contrast

Soft vs. Contrast

High-contrast text was tested against softer colors. The high-contrast page drew 0.9 percent more queries per user and 3.1 percent more ad clicks.

Images: Testing Microsoft’s Experimental Platform

Feedback A vs. Feeback B

Feedback A vs. Feeback B

Feeback A puts everything together while Feedback B is a two-step process–a question following the ratings. Feedback B got more than double the response rate.

Images: Testing Microsoft’s Experimental Platform

Feedback C

Feedback C

This feedback is also a two-stage process. It outperformed Feedback B by 3.5 times.

  • CXO
  • Account Information

    Share with Your Friends

    Images: Testing Microsoft’s Experimental Platform

    Your email has been sent

Share: Images: Testing Microsoft's Experimental Platform
Image of Andy Smith
By Andy Smith
  • Account Information

    Contact Andy Smith

    Your message has been sent

  • |
  • See all of Andy's content

TechRepublic Premium

  • TechRepublic Premium

    Hiring kit: Automation specialist

    PURPOSE Recruiting an automation specialist with the right combination of technical expertise and experience will require a comprehensive screening process. This Hiring Kit from TechRepublic Premium provides a flexible framework your business can use to find, recruit and ultimately hire the right person for the job. From the hiring kit: DETERMINING FACTORS, DESIRABLE PERSONALITY TRAITS ...

    Published:  March 27, 2023, 12:00 PM EDT Modified:  March 28, 2023, 12:48 PM EDT Read More See more TechRepublic Premium
  • TechRepublic Premium

    Hiring kit: Cybersecurity engineer

    PURPOSE Recruiting a cybersecurity engineer with the right combination of technical and industry experience will require a comprehensive screening process. This hiring kit from TechRepublic Premium provides a flexible framework your business can use to find, recruit and ultimately hire the right person for the job. From the hiring kit: DETERMINING FACTORS, DESIRABLE PERSONALITY TRAITS ...

    Published:  March 27, 2023, 12:00 PM EDT Modified:  March 28, 2023, 12:18 PM EDT Read More See more TechRepublic Premium
  • TechRepublic Premium

    Hiring kit: Business information analyst

    PURPOSE A large segment of internal and external modern business now takes place on computers, tablets and other mobile devices. The information technology infrastructure that has sprung up around that business can be daunting in its scope. This Hiring Kit from TechRepublic Premium provides an adjustable framework your business can use to find, recruit and ...

    Published:  March 26, 2023, 12:00 PM EDT Modified:  March 28, 2023, 12:51 PM EDT Read More See more TechRepublic Premium
  • TechRepublic Premium

    Grant writing guide

    PURPOSE This grant writing guide from TechRepublic Premium will help you identify a grant that fits your idea, build a team to help you compile a proposal, and set a timeline to meet the application deadline. From the guide: BEFORE YOU START This list of steps will help the grant writing process go more smoothly. ...

    Published:  March 26, 2023, 12:00 PM EDT Modified:  March 27, 2023, 6:00 PM EDT Read More See more TechRepublic Premium

Services

  • About Us
  • Newsletters
  • RSS Feeds
  • Site Map
  • Site Help & Feedback
  • FAQ
  • Advertise
  • Do Not Sell My Information

Explore

  • Downloads
  • TechRepublic Forums
  • Meet the Team
  • TechRepublic Academy
  • TechRepublic Premium
  • Resource Library
  • Photos
  • Videos
  • TechRepublic
  • TechRepublic on Twitter
  • TechRepublic on Facebook
  • TechRepublic on LinkedIn
  • TechRepublic on Flipboard
© 2023 TechnologyAdvice. All rights reserved.
  • Privacy Policy
  • Terms of Use
  • Property of TechnologyAdvice