Collect metrics that will provide a basis for fundamental improvements

A project manager is asked to collect metrics to measure customer satisfaction. Here's some advice on the right kind of metrics to collect if you want to make improvements.

Each week, project management veteran Tom Mochal provides valuable advice on planning and managing projects. He first describes a common problem scenario based on a real-life situation and then offers a solution using practical project management practices and techniques.

Terry is managing and working with one other person on an unusual small project. He was asked to put metrics in place to measure the overall customer satisfaction of the manufacturing department users when they call the help desk. Terry first gave me some background about why this effort was under way.

The IT department at Terry’s company had just installed manufacturing software at the new plant. “It seems that there are an unusually large number of problems,” Terry said. “The IT team has resources that are dedicated to getting the bugs ironed out, but we could be dealing with a high number of support calls for many months.”

“Why does the client think that gathering metrics will help?” I asked.

The dilemma
“The client is not happy with the problem resolution process,” Terry told me. “They have difficulties getting calls logged in the help desk. Then they think they’re waiting too long for return calls. They’re also not always happy with the resolution of the problems.”

“What does the help desk say?” I asked.

“Actually, this gets at one of the primary reasons for setting up the metrics process,” Terry said. “The help desk and the IT support group think they’re responding in a timely manner. However, no one knows the facts because we don’t have any quantifiable numbers.”

Terry gave me the initial list of metrics he was proposing to gather. “We want to collect information on the time to reach a live person on the help desk, the time until an initial follow-up call from the support group, the time to resolve the problem, and customer survey metrics on the professionalism, knowledge, and courtesy of the help desk and support people.”

“That sounds like a great start,” I said. “But you’re missing a set of metrics that is even more important.”

Mentor advice
On the surface, the metrics that Terry is proposing look fine. They will help quantify the level of service that the help desk and IT support staff are providing. But these metrics don’t get at the root causes of the problems the client is experiencing. Terry is only proposing to measure how quickly the problems are resolved once they surface. To get at deeper problems, he needs to ask questions from the perspective of clients. Yes, they want their problems resolved quickly. However, I’m sure what they really want is to not have the problems to begin with.

Let’s assume that if valid statistics were available today, they might show that the client was reporting 50 problems per week, with an average resolution time of 24 hours. If the focus is only on turnaround time, you can imagine that in a few weeks, the average response time might be 12 hours. This would certainly be a better situation for the client. However, if there are still 50 problems reported per week, the client will remain dissatisfied.

From the client’s perspective, other metrics should be gathered as well, such as:
  • The number of problems reported per day and per week.
  • The number of problems that are identical to problems previously reported.
  • The types of problems grouped into categories, for further root cause analysis (i.e., Pareto Analysis).
  • The severity of levels of the problems (focusing on eliminating major problems, then medium, and then minor).
  • The impact, in terms of lost hours, of the problems.

My advice to Terry is to keep the metrics he’s proposing but to add more metrics in terms of client impact. An old adage says that what gets measured, gets done. I don’t want Terry to focus exclusively on the customer service level after a problem surfaces, because doing so would direct attention away from heading off problems before they arise. Instead, the goal should be to reduce the number and severity of the problems and reduce their impact on the client. In addition to that primary focus, when a problem arises, it should be resolved as quickly and painlessly as possible.

Project management veteran Tom Mochal is director of internal development at a software company in Atlanta. Most recently, he worked for the Coca-Cola Company, where he was responsible for deploying, training, and coaching the IS division on project management and life-cycle skills. He's also worked for Eastman Kodak and Cap Gemini America and has developed a project management methodology called TenStep.

Editor's Picks

Free Newsletters, In your Inbox