By Seth Gordon
2/15/00

Before starting the testing process, it is important to define the broad goals of the evaluation. The question to answer is, Why are we
running this usability evaluation? Common motivators for usability
evaluations include checking whether a user can collect specific
information or perform certain tasks, such as completing a purchase
transaction from start to finish. You can also use the test as a tool to
resolve areas in which the clients and the developers have differing
approaches to treating a specific component; for example, whether a
pull-down menu works better than radio buttons do.

What Are You Testing?
Once you’ve defined the broad goals of the test, you’ll need to decide what
exactly you’re testing. A usability evaluation can take place anytime
within a development cycle, so it’s possible to test the site in just about any form:

  • Paper prototype: Sketches or printouts of potential page
    layouts. Respondents may need to use a bit of imagination when assessing
    early-stage deliverables, but it’s an excellent way to test page layouts.
  • Design comp: Visual designs or mock-ups of proposed pages.
  • Wire frame: Early versions of a site with limited depth and
    functionality. Useful for testing processes (registration, purchase, and so
    on) and site flow.
  • Live site: Publicly available version of a functioning site.

After deciding what to test, set a date on which all edits to the test
materials must stop. It’s the same idea as a code freeze; the
development team needs time to get the test materials in order.

What Are Your Goals?
Now that you know what you’re testing, you’ll need to decide what the
primary data collection goals are. For example, are you assessing
respondents’ expectations and perceptions about the site, or are you
collecting their feedback on what works well and what doesn’t? You’ll also
need to decide if the results will come primarily through observation, or
if the evaluation will be designed to collect quantifiable feedback, such as
the successful completion rate of a specific test. Whatever your goals, make sure the data points are predefined and easily
collectible.

What Is Your Client’s Expectation?
Inform the internal team and the client about how the evaluation is going
to run and how you’ll incorporate the findings into the development
process. This is an excellent opportunity to get internal buy-in and
derail future attempts to question the test methodology. Often the
final solutions to usability concerns come from the development team as a
whole, not just the usability engineer. Make sure people understand how
final decisions will be made and how they will impact the rest of the project.

A common misconception is that usability evaluations run the same way as
marketing research groups. They don’t. Head off this confusion at the start
of the project. One notable difference is the sample size, which is much
smaller, especially for discount usability evaluations. Market research
focus groups often seek to collect perceptual information and establish
statistically relevant trends; that can’t happen with a sample size of
five to six people.

Note: The term client is used in this article to describe the entity
that owns the site being tested. It can describe an internal team or a
paying customer.

Seth Gordon is a frequent contributor to CNET Builder.com. In his spare time, Seth enjoys touring national parks and restoring mechanical antiques.