CXO

User surveys require good planning and attention to detail

Developing a help desk user survey can be challenging for a new manager. Take some tips from a management guru on how to create and distribute a successful survey that reveals useful data.


TechRepublic member Will Lockett recently e-mailed us about wanting to conduct a survey of help desk users. Will operates a help desk that serves 120 users for a company in California. He is fairly comfortable with the level of support being provided, but wants to know if there are ways the help desk team can improve. Will also wondered if there are sample surveys that could be used for his evaluation process.

There is really no one best way to evaluate work team and help desk performance. Effective managers will incorporate a variety of processes into their evaluation process, including hallway conversations, user surveys, and formal meetings.

User surveys are certainly an excellent way to determine the level of satisfaction among those employees who have to rely on the help desk each day. However, it is important to approach the use of user surveys carefully and with considerable planning. If financially feasible, it can be helpful to consult with a trained evaluator on the development of a user survey. However, this may not always be practical or possible. Here are some suggestions for developing a successful user survey process.

Determine the primary purpose for the survey
What is the one thing that is most important to accomplish with the survey? The answer to this question will serve as the foundation for the entire evaluation process. For example, if you just want to know how users feel about your team’s technical support, you will design and conduct the survey to prioritize that type of information. If you are looking to gather input about how the LAN could be improved or to determine the feasibility of incorporating new technology, the survey will look very different. Having a clear idea of what you want to accomplish will strengthen the survey process and make it easier to sell the idea to senior managers.

Keep it simple
In most cases, a clear and simple survey instrument will result in more accurate and usable results. There is often a tendency to intermingle more than one concept into a question. This can be a problem because users will be confused about how to answer the question. Further, it will be difficult to interpret the data that you collect. For example, if you ask a question such as, “Was the technician who handled my issue competent and pleasant in his approach toward me?” you are actually asking two different things. The technician may have been very competent but miserable to work with (or vice versa).

Decide on the best format for the survey
Generally, surveys that have check boxes for answers are easier to complete by users. However, this type of survey doesn't allow for more in-depth responses. Open-ended survey questions that require written answers provide richer information about user views but are more difficult and time consuming to complete. A compromise to these approaches is to use a Likert scale survey that allows the user to choose among several responses to each question (e.g., ranging from strongly agree to strongly disagree). You can then provide for an opportunity for each participant to add a comment for each question and at the end of the survey. This approach allows for ease of use by participants while providing an opportunity for additional comments if desired.

Cover your bases
It's essential to brief stakeholders in the help desk about the survey and why you are conducting it. A major mistake would be to conduct a survey of users without notifying and involving their supervisor(s). Senior managers would also want to be involved (or at least be informed) since they may be approached by external sources such as journalists or other managers within the organization about the survey.

Be prepared for the information you receive
Map out a strategy for incorporating desired changes into the help desk based on the results of the survey. Most users will want to know the results of the survey, so you need to develop a dissemination plan for the information you receive. You will more than likely lose credibility with users if you ask for their opinion of the help desk’s support services and then do nothing to address the concerns that are raised.

Don’t overanalyze the survey results
It's often sufficient to compile the check-box responses to each question and then list any written responses received for reporting purposes. If you wish to analyze survey responses to any extent, it's best to consult with a trained evaluator. Misinterpreting survey results is very easy, and a trained evaluator will know the pitfalls of the process. Also, you must protect the privacy of survey participants unless you receive written permission to use their names.

Example scenario
To gain a better understanding of these issues, consider the following scenario: Becky is a new IT manager and is responsible for maintaining a help desk that serves 100 users. Becky was told, when hired, that she would be responsible for improving the technical support provided by her team. Upon starting her new position, she learned that there was no data available showing problems with the team.

Becky decided to set up a user survey process where users would receive a short survey by e-mail with questions about the efficiency and effectiveness of help desk support. She was not sure how to proceed, so she approached a colleague in the program evaluation unit for help. Her colleague offered to help but asked Becky to approach his supervisor first to get permission. Becky spoke to the supervisor and offered her unit’s help in the process. Becky was asked to provide the following information:
  • Purpose of the survey
  • Type of information being sought
  • Preferred design
  • Plan for dissemination of the survey results

Becky provided this information and also notified people who she believed would be impacted by the survey. These people included the users’ supervisors, senior program management, and the human resource director. Becky’s colleague helped her develop a short Likert scale survey with space for additional comments for each question. The survey was provided to users online, and Becky’s team received a 90-percent response rate.

Becky used the results of the survey to initiate changes in help desk processes and as a baseline for establishing an evaluation system for her team. Her colleague assisted her in developing a new online survey that would be sent to users whenever they requested assistance from the help desk. Becky was very pleased with the process because she was able to obtain good baseline data for her team, a way to track user satisfaction over time, and good relations with the program evaluation unit as well as senior management.

The moral of this story is that user surveys can be very helpful in determining work team performance. However, surveys do require good initial planning, sensitivity to the impact the process will have on others, and willingness to use the information in constructive ways.

Final thoughts…
A well-planned and implemented user survey can be an excellent evaluation tool for a help desk manager. However, it is important to have a clear understanding of its purpose and to maintain high ethical standards in regard to handling the information being collected. Remember that user survey results must be divulged, in some cases, for Freedom Of Information requests (for federal and state government) and for many legal proceedings such as personnel actions.

IT managers who are interested in incorporating user surveys into their team’s evaluation process should read Small-Scale Evaluation by Colin Robson (2000) and How to Conduct Your Own Survey by Priscilla Salant and Don A. Dillman (1994). There are also a wide variety of research consulting firms that can provide assistance as well as Web-based services that offer sample surveys and assistance with survey development.

New manager questions
Steven Watson has 10 years of IT management and consulting experience and has developed an understanding of how the issues faced by IT managers differ from those of their nontechnical colleagues. As a new tech manager, do you have a question you’d like him to address? Send it to us via e-mail, or post it in the discussion below.

 

Editor's Picks