CXO

User testing: Defining your goals

Before you start testing your users, first define your evaluation's goals. Read on to find out what questions you should answer.

By Seth Gordon
2/15/00

Before starting the testing process, it is important to define the broad goals of the evaluation. The question to answer is, Why are we running this usability evaluation? Common motivators for usability evaluations include checking whether a user can collect specific information or perform certain tasks, such as completing a purchase transaction from start to finish. You can also use the test as a tool to resolve areas in which the clients and the developers have differing approaches to treating a specific component; for example, whether a pull-down menu works better than radio buttons do.

What Are You Testing?
Once you've defined the broad goals of the test, you'll need to decide what exactly you're testing. A usability evaluation can take place anytime within a development cycle, so it's possible to test the site in just about any form:

  • Paper prototype: Sketches or printouts of potential page layouts. Respondents may need to use a bit of imagination when assessing early-stage deliverables, but it's an excellent way to test page layouts.
  • Design comp: Visual designs or mock-ups of proposed pages.
  • Wire frame: Early versions of a site with limited depth and functionality. Useful for testing processes (registration, purchase, and so on) and site flow.
  • Live site: Publicly available version of a functioning site.
After deciding what to test, set a date on which all edits to the test materials must stop. It's the same idea as a code freeze; the development team needs time to get the test materials in order.

What Are Your Goals?
Now that you know what you're testing, you'll need to decide what the primary data collection goals are. For example, are you assessing respondents' expectations and perceptions about the site, or are you collecting their feedback on what works well and what doesn't? You'll also need to decide if the results will come primarily through observation, or if the evaluation will be designed to collect quantifiable feedback, such as the successful completion rate of a specific test. Whatever your goals, make sure the data points are predefined and easily collectible.

What Is Your Client's Expectation?
Inform the internal team and the client about how the evaluation is going to run and how you'll incorporate the findings into the development process. This is an excellent opportunity to get internal buy-in and derail future attempts to question the test methodology. Often the final solutions to usability concerns come from the development team as a whole, not just the usability engineer. Make sure people understand how final decisions will be made and how they will impact the rest of the project.

A common misconception is that usability evaluations run the same way as marketing research groups. They don't. Head off this confusion at the start of the project. One notable difference is the sample size, which is much smaller, especially for discount usability evaluations. Market research focus groups often seek to collect perceptual information and establish statistically relevant trends; that can't happen with a sample size of five to six people.


Note: The term client is used in this article to describe the entity that owns the site being tested. It can describe an internal team or a paying customer.

Seth Gordon is a frequent contributor to CNET Builder.com. In his spare time, Seth enjoys touring national parks and restoring mechanical antiques.

Editor's Picks