CXO

Talking Shop: Survey your end users the right way

Craft a user survey that will render meaningful results.


Is your help desk staff doing a good job? Do they solve user problems in a timely manner? Do your end users like the people on the tech support team? There’s only one way to find out the answers to these questions: Ask your users.

Whether you support 200 or 2,000 end users, you should periodically survey those users to find out what they think of the level of tech support they’re receiving.

Last fall, I wrote a customer satisfaction survey for over 1,600 end users. After the survey was posted on the company’s intranet for two weeks, I analyzed the results of the survey and presented a report to the people in charge of the help desk. I’d like to help you jump-start the process of surveying your own end users by sharing some of the questions I wrote for my client’s questionnaire.

Get the big boss to sign the invitation
Before I get to the survey questions, let’s talk about one of the biggest challenges you face when you attempt to survey your users—motivating them to take 10 minutes to answer the questions.

In this case, I composed an e-mail message to be sent to all end users in the organization. However, the person who hired me didn’t sign that message—his boss (the CEO) did. Our thinking was that the higher the rank of the person making the request, the more likely users would be to click through to the survey.

When the e-mail invitation went out, the subject line was: “How are we doing?” Here’s an edited version of the invitation sent via e-mail:

"Last year, we launched an online help site (the Tech Support page) which provides answers to frequently-asked-questions about computers, and which allows you to initiate trouble tickets via a Web-based help desk system. In the last nine months, over 1,000 of you have used the help desk system to generate over 5,000 help desk requests.

"This year, we're conducting a follow-up customer satisfaction survey to help us provide the best possible technical support and training for our employees. Please help us meet our goal of 100 percent participation and take 10 minutes to fill out this year's online survey." [signed by the CEO]

The phrase “fill out this year’s online survey” was a hyperlink to the survey on the intranet. For the benefit of those users who skim text, we also included at the bottom of the e-mail the following bullet points. The last item also included a hyperlink to the survey page.
  • WHAT: Technical Support Customer Satisfaction Survey
  • WHO: Every employee
  • WHY: To tell us how we're doing and how we can improve technical support and training services
  • WHEN: The survey opens Monday, September 10 and closes Friday, September 21.
  • HOW: Click this link to display the survey.

For the record, the in-house development staff created the survey page, gathering the results using an Oracle database and exporting the results to Access. I used the Access tables to generate the summary results on which the final report was based.

The survey questions
When you’re ready to start writing questions, consider what broad areas of help desk services you want to cover in your survey. My client wanted questions that measured success in three areas:
  • Users’ satisfaction with the help desk in general
  • Users’ satisfaction with the speed with which issues are addressed and resolved
  • Users’ satisfaction with the technical training available to them

Before you write a single question for the survey, review the entries in your help desk database and look for trends. In my client’s case, one particular statistic stood out: Over 70 percent of trouble tickets were resolved the same day they were opened.

My client suspected, however, that users would be more likely to remember (and complain about) the issues that took longer than a day to resolve. Therefore, we made sure to include questions about how long users perceived that it took to resolve most help desk issues.

I wrote over 250 questions from which the client culled about 30 for the final survey. The first set of questions was an attempt to establish a profile of the person answering the survey.

The first question was: "Which term best describes your computer skills?" The possible answers were: Expert, Above Average, Average, Need Training, and None.

Next, we asked the following yes or no questions:
  • Do you use the Tech Support intranet Web site as your portal or home page?
  • Do you know who your department’s technical point of contact is?
  • Do you know who your local IT person is?

The client suspected that most users were relying on the telephone to request help. To address that suspicion, the next question was: “To request help with my computer, the method I most frequently use is: Help Desk (intranet), Telephone, E-mail, In Person, Ask Someone Else.”

Satisfaction with service
Next, we tried to find out what users thought of the level of service being provided. The possible answers to the following questions were: Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree:
  • The last time I requested computer help, I was completely satisfied with the overall quality of the service I received.
  • I am satisfied with the way the help desk's intranet Web site works.
  • The last time I requested computer help, I was completely satisfied with how long it took to resolve my problem.
  • I am satisfied with the response time to printer problems.
  • The last time you requested help, how satisfied were you with the way the person assigned communicated with you?
  • How satisfied were you with the time it took to first contact you about your request?
  • How satisfied were you with the total time it took to resolve your request?

Satisfaction with people
My client’s help desk staff supports end users in more than one location, and the feeling was that users would probably give higher marks to their local IT contacts than they would to the overall corporate IT staff. To find out, we asked users to rate specific groups as Excellent, Above Average, Average, Below Average, or Poor/Do not use. That section of the survey started out with the sentence, “Please rate your perception of the service you receive from the following groups of people” and included the following entities:
  • The main agency’s IT staff
  • Your local technical point of contact
  • Your division or department IT staff
  • [Company X] (a local hardware maintenance contractor)

We also asked: “The last time you requested help, how satisfied were you with the way the person assigned communicated with you?” The possible answers were: Very Satisfied, Satisfied, Neutral, Dissatisfied, and Very Dissatisfied.

Satisfaction with infrastructure
We wanted to find out whether users were satisfied with the equipment provided to them, so we asked a series of questions for which possible answers were: Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree. We asked users to rate their level of agreement with the following statements:
  • I have all the computer software and hardware I need to do my job.
  • I am satisfied with my own knowledge of and experience with personal computers.
  • I am satisfied with the agency’s ability to protect my computer from e-mail viruses.
  • I am satisfied with the IT Support team's overall level of knowledge and ability to resolve problems.
  • I am satisfied with the Technology Support page's virus updates, password changes, e-mail tips, and answers to frequently asked computer questions.
  • The agency has enough IT support professionals to meet the needs of its employees.
  • I am satisfied with the agency’s knowledge of my technology needs.

After that section, we gave users the chance to enter free-form text in responding to this question: “Enter your comments about IT support. (You have a maximum of 1,024 characters.)”

Looking for templates?
For more ideas to help you create a customer satisfaction survey for your organization, download these three survey templates from project management mentor and TechRepublic contributor Tom Mochal. The first survey solicits feedback from end users of support, the second focuses on a non-support service organization, and the third is aimed at customer management.

Satisfaction with training
The final section of the survey covered user satisfaction with the courses provided by the agency’s full-time trainer. (The possible answers are listed in parentheses.)
  • How many IT or computer training classes have you attended (answer restricted to integers)?
  • The last time I took a computer class, I was satisfied with the training I received (Strongly Agree through Strongly Disagree).
  • How important do you feel technical training is for your career (Very Important through Very Unimportant)?
  • I am satisfied with the number, frequency, and type of computer classes offered by the organization (Very Satisfied through Very Dissatisfied).
  • Enter your comments about training. (You have a maximum of 1,024 characters.)

Calculating survey results
I could write an entire column on how I tabulated the results of that survey, but most of the reports I generated were based on common sense. The first report the client requested was simply the raw numbers. That is, on a question-by-question basis, how many people clicked which answer? For instance, with Yes or No questions, the results took this form:
Yes: 750 (75 percent), No: 250 (25 percent)

I took a slightly different approach with the free-form text responses. I threw out the responses that had no statistical or substantive value—such as “none,” “don’t know,” and “n/a.” I sorted the rest of the answers by the user’s department, which was stamped in the records based on the user’s logon when the survey was filled out. I pulled some of the text entries and put in a special report labeled, “May deserve immediate follow-up.” Based on his discretion, the CIO forwarded some, but not all, of the comments to the appropriate department heads.

Finally, the CIO had me do a series of reports based on subsets of the results. For instance, we ran separate summaries of responses to all questions for users who rated themselves as having “Expert” knowledge of computers compared to the folks who rated themselves as “Need Training.”

Add your two cents
Have you surveyed your customers lately? To share the results with your fellow TechRepublic members, post a comment below or write to Jeff.

 

Editor's Picks