Think before you measure

Proceed with extreme caution when it comes time to evaluate the performance of your staff members. Artner's Law explains why it is important to think twice and measure once when deciding how to track an employee's performance.

Data. For the last 50 years, at least, what every manager and executive wants is data—quantifiable information that you can use to make decisions. While many employees think their supervisors make decisions capriciously or at the spur of the moment, most of the IT managers I know are committed to letting data drive most decisions.

This factor explains the rise of CRM (customer relationship management) software and systems, which promise to provide you with immediate feedback from your customers so that you can react quickly to their needs and desires.

Who could argue against that? Not me, exactly. But in this column I am going to show that you need to be careful when deciding what to measure, especially when you are measuring the performance of your employees.
In our Discussion Center, we're talking about how to measure employee performance effectively. To add to this discussion, post your comment to this article. Each week, the person who provides the best feedback to an Artner's Law column will win a free TechRepublic coffee mug.
To find out who each week's winner is and to subscribe to Artner's Law, sign up for the TechRepublic TechMail now.
Incentives matter—and so do measurements
When it comes to compensation, everyone agrees that incentives matter. All other things being equal, we would expect that someone who earns x per year would be more motivated if his or her salary was raised to 2x per year.

This is because human beings don’t exist in a vacuum. We react to changes in our surroundings. In fact, drug studies have to discount the “placebo effect,” where patients report that an experimental drug is working—even though they are taking sugar pills as part of the control group.

When it comes to measuring performance, remember the law of unintended consequences. The economist Thomas Sowell, in a speech on C-SPAN years ago, illustrated this point with an example from the former Soviet Union.

It seems that for a time, the managers of nail factories in the USSR were judged based solely on the sheer weight (in tons) of nails produced each year. Size or type of nail (to say nothing of quality) was irrelevant—the only question factory managers had to answer was, “How many tons of nails did you make this year?”

An unintended (though predictable) consequence of the focus on the weight of produced nails was a nationwide shortage of small nails. Since the factory managers were judged by raw tonnage, they focused on making really big nails. After all, it’s much easier to drive up the total weight of produced nails by making smaller quantities of really big nails instead of lots of little nails.

That might have made sense to a nail factory manager, but it was small consolation to the politburo member who had to use a 10-inch nail to hang a picture in the living room of his dacha in the Crimea.

After a time, the powers that be changed the measuring scheme. Sheer weight of produced nails was out as a metric. The new focus was on the number of nails produced each year.

Well, you can guess what happened next. Sowell said that it wasn’t too long before you could hear the sound of chandeliers crashing onto ballroom floors all across the Soviet Union, since the nail factories produced huge quantities of very tiny nails—and no large ones.

Measuring behavior changes behavior
What does this mean for the IT manager? Consider the help desk. How do you measure the performance of your support technicians? Some companies look at the number of tickets cleared by each technician. These companies are pleased when the number of tickets cleared rises shortly after the new measurements begin.

However, many of these same companies find that the support techs are rushing through each ticket and not giving adequate service, since they want to clear the ticket and move on to the next call.

On the other hand, if you judge performance strictly based on the results of end-user satisfaction surveys, your support techs might dawdle too long on each call.

Of course, this isn’t an insurmountable problem. Most companies solve it by creating a benchmarking system, where calls are divided into categories, with a different completion time assigned to each category. Most companies also provide opportunities for end-user ratings, so that in the final analysis, support techs are judged on a variety of relevant criteria.

These same sorts of issues face managers of development teams. How do you measure the performance of a programmer? Do you focus on the sheer amount of code? Or is it the ability to hit project deadlines? What about the quality of the produced code? How do you factor in the need for debugging or the ability to work with internal and external clients?

Focusing on any one of these criteria to the exclusion of all others can result in a programmer who excels by your chosen metric but still isn’t a productive member of your team. For example, how many times have you heard about development teams that pushed an application into production too early just to get a deployment bonus? For that matter, if you judge a programmer based on the ability to hit a deadline, you create an incentive for the programmer to pad his or her estimates.

What to do?
Faced with the quandary of accurately measuring employee performance, IT managers should follow this strategy:
  • Benchmark. Now this is one of those terms that means different things to different people. In this case, I’m advocating that you find a basket of metrics and not concentrate on any single factor.
  • Use your best people as guides. Often, it’s easier for a technical manager to identify his or her best employees than it is to create benchmarks. Use your best performers to help. Look at your star people and use their productivity as a standard for the rest of your group.
  • Be prepared to change your metrics. Remember, you’re making (at best) an educated guess, especially when you first roll out a benchmarking strategy. Who was it that said no battle plan ever survived contact with the enemy? Take that same attitude with your metrics. Explain them to your people and be open to the need to change metrics if they don’t adequately measure (and improve) employee performance.
  • Don’t dwell on the results. Remember, benchmarking and other metrics are just tools to help you be a better manager. They won’t do your job for you.

Good luck!
When you evaluate an employee, do you ask the employee’s coworkers to evaluate this person? Have you ever seen a manager evaluate a staff member in a way that hurt morale or damaged workflow? To add to this discussion, post your comment to this article.

Editor's Picks

Free Newsletters, In your Inbox