How to conduct your own website usability testing

Ryan Boudreaux details all the pieces that should be included in a plan to conduct your own website usability testing.

What grade would your website earn if it took a usability test? You may not realize it, but your website is getting tested every day by your users, and within the first eleven seconds of their visit they have already made a decision to either stay or leave. Usability testing can be done in-house, utilizing your own resources, or it can be outsourced to a third-party firm that specializes in the task of getting websites back on track. In any situation, you will need a solid checklist as a guide, which will eventually become a catalog of your required updates.

Not only is usability testing important before the launch of a new or redesigned website, it is just as important for existing sites that have not been updated for quite some time. Customer expectations and business models change over time, and you need to ensure that your website is keeping up with both. A rapidly changing and evolving business environment calls for a website that meets the usability needs of both the customer and the business; finding the right balance can mean great success for the client, customer, and the organization.

Doing your own usability testing

Maybe your company does not have a large budget or the time to hire out a third-party usability testing organization -- no problem. All you need is a usability plan, and then execute it getting the input you need to make decisions based on the results of the testing, and then set an action plan to tweak, modify, or overhaul the entire website based on your findings.

The usability testing plan

First you should get with your marketing department, management, or customers -- the people who give you the content for their website. If the web development team or webmaster is responsible for the site content, then it might take much less time to get a plan on paper; however, on the flip side, if you have a few folks who can be recruited into the "Usability Testing Team" then getting help is always better than doing it all on your own.

The intention of the usability testing plan is to record and outline the steps of what you are going to accomplish, how you are going to perform the testing, what metrics you are going to capture, the number of participants you are going to employ, and what testing situations you will use. A very good resource for what the typical usability plan should include and the major elements of the plan can be found at This list includes important elements of a testing plan such as scope, purpose, logistics, equipment, scenarios, and metrics identified in the plan.

Typically, the usability guru (you) meets with the rest of the team to decide on the major components of the plan. Often, you or designated team members then draft the plan, which then gets passed around to management/customers and the rest of the team. Once everyone has commented and a final plan is agreed upon, then a revised written plan is completed to reflect the final decisions.

Conducting the testing

Now that you have the testing plan laid out with all the approvals, it's time to implement the testing. Make sure your test participants are similar to the types of people who will use your web site; for example, if your website caters to youth and teenagers, then you need to recruit similarly aged participants in your testing. Make sure you have the testing scheduled to coincide with space requirements and availability, location logistics, equipment, and have your testing scenarios set up and ready before your participants are recruited.

Make sure your participants are comfortable, and have a facilitator explain the testing session, allowing them to ask questions. Assign the tasks and scenarios to each participant. Explain that testing ends upon completion or after a certain time, yet allow participants enough time to complete the selected tasks. At the end of the session, the facilitator should ask any final subjective questions, and thank them for participating.

Getting the results

The test metrics collected in the usability testing should include several objective and subjective measurements, including:

  • Whether the participant was able successfully to complete the selected tasks or test scenarios as outlined on the checklist. This could be obtained with a simple multiple-choice questionnaire for participants.
  • The time that it takes a participant to complete a task is a good measure of usability.
  • Subjective measurements allow participants a self-assessment for levels of satisfaction including ease of use and level of effort to complete their assigned tasks -- giving them a 5 to 7-point Likert rating scale.
  • Allow participants to comment on their likes and dislikes, and offer any recommendations for improvements.

Analyze the results

Once all the usability testing is completed, it is now time to collect and compile the data you have collected. Quantitative data you should have collected include success rates, task time, error rates, and satisfaction questionnaire ratings; qualitative data includes observations about the testing that participants shared about problems they experienced and answers to open-ended questions.

The information collected from the quantitative data should allow you to perform certain important calculations with important results such as the percentage of participants who completed tasks, the average time it took to complete a task, and the frequency with which participants ran into problems. Another way the data can be presented is to have it split out into demographic categories, where each target audience may have differing results. The qualitative data should be collected and compared to find any trends that were recurring for participants.

Reporting the results

Make sure your report is just as solid as the hard work you've already put into your planning and execution of the usability testing. Remember, you want to improve the website based on your findings; a solid report will be your guide to getting the website back on track. And if you have to sell the idea to upper management or marketing, you may also need to put together a solid presentation.

Be sure to present your findings in a leveraged approach where the results are categorized based on level of importance with the most pressing issues given a higher priority and less pressing issues given a lower priority. Then group the results based on each level of concern, concentrating your report on the high priority topics first, then the medium priority results, and so on throughout the report.

The report should be outlined and begin with a background summary on why the testing was needed, and then move onto the methodology, using charts and tables to highlight certain aspects of the testing sessions, participants, metrics collected, and overview of the tasks. Next, show the results of the data. This is the real meat of the report and presentation. Be sure to report any positive findings, highlight any opportunities for improvement, and suggest ways to leverage the good to make the site even better.

Making the changes

Once the findings are presented, it is time to link them to the recommendations on how the site needs to be modified based on the results. The recommendations should be the last part of the final report, and they then become talking points for making the final decisions on what needs to change to make the site a more usable and pleasing user experience. All recommendations may not be able to implemented, due to a budgeting issue or scheduling. If all the recommendations cannot be implemented, then go back to the priority list and make sure the high priorities are put into action first, and then try to implement the medium level items, and so on. At least the glaring design flaws will be fixed, your web site will have more return on its investment, and your users will be happier for it too.

The next piece will cover hiring outside resources for your usability testing.

By Ryan Boudreaux

Ryan has performed in a broad range of technology support roles for electric-generation utilities, including nuclear power plants, and for the telecommunications industry. He has worked in web development for the restaurant industry and the Federal g...