CXO

The metric system: Keeping it real

You know you're good, but can you prove it? When it comes to development, you need to be sure the metrics used in your shop provide an accurate and balanced picture of what you do. Read these tips on making sure management knows your true batting average.


In seasoned development shops, managers typically use a variety of metrics to determine the quality and value of an application. These measurements may also be used to measure individual or team productivity. However, one set of statistics can't properly reflect a developer's productivity. As well, the true value of an application and its underlying code can't be explained via a set of statistics that didn't take into consideration the needs of the user.

Yet sometimes you have no direct control over these stats. And if ill-defined or incomplete metrics are used to establish raises, bonuses, or promotions, it can leave you feeling powerless and discouraged. On the other hand, a well-developed performance metric can be a positive motivator. The key is knowing how to determine whether or not the metrics to which you’re being held are working for you.

Are you in the driver's seat?
There are three questions you should ask when looking at any performance statistic:
  1. What is being measured?
  2. How is it being used?
  3. How much control do you have over the outcome of this measurement?

Alexander Hiam, author of the book Motivating and Rewarding Employees: New and Better Ways to Inspire Your People, suggests managers “switch from tracking a department’s overall productivity to tracking the productivity of each individual in the department so each can be held accountable for his or her contribution to production.” Additionally, he says the “potential to bring about results through your own efforts” is an important part of task clarity, which he considers vital for motivation and performance. Fast and accurate feedback is the other aspect of task clarity that employees in the study found important to high performance. If you are subject to measurements that penetrate only team level performance, you may want to work with management to increase the individualization of those measurements. For example, if your company only looks at the number of programs put into production, you might consider looking at some measures (e.g., turnaround time on bug fixes, number of programs produced, function points, etc.) on a per-programmer basis. This will help provide you with more valuable feedback and ensure that your hard work is recorded for management to see.

Do you have balance?
A single set of measurements is probably not an effective way to gauge a developer’s or an application’s productivity, but a combination of different statistics provides a well-rounded and therefore more accurate look at the big picture. Think of the story of the three gentlemen who happened upon an elephant but were only able to see part of the animal through a hole in the fence. Each based his perception of what the animal was on his own observation of just the part of the animal he could see through the hole. Because the elephant was large, as elephants are prone to be, none of the individual observations were correct. Taking the sum of the parts, however, they would have come up with a more accurate account.

The same is true of your work. A measure showing the number of lines of code produced is certainly a measure of productivity. But as you know, that's not the whole story; a thousand lines of garbage may as well be zero lines. So adding measures of quality and features might help management to have a better view of your accomplishments.

What can you do?
There are some questions to ask when analyzing the development metrics that really make a difference. It is important to note that the metrics you'll be interested in during development are going to be different than the metrics you'll be concerned about once your application is in production and maintenance mode. I will explain production and maintenance metrics in a future article.
  • Are metrics used for programs in maintenance and those used for new development approached differently? They should be.
  • What about peer code reviews? Do you have clear guidelines for what coding techniques qualify as “good”? Are flaws in business logic tracked in some way? Do you provide feedback to and receive feedback from your peers to encourage all team members to perform to the standard?
  • Have you updated your measuring process as you’ve changed development techniques? If you’ve moved from third- or fourth-generation language development to object-oriented development, have your old measurement techniques been replaced? And if you’ve changed processes, say from waterfall to extreme programming, how do your measurements account for that?
  • Is that lines-of-code statistic working? This metric can encourage misuse and should not be used to assess your performance. It might be useful as a measure of program complexity, but frankly, it really isn’t very good for that either.
  • Does a reuse measurement exist? If you are evaluated based on the number of lines of code you produce, this is in direct conflict with any object-oriented design (OOD) metrics that encourage code reuse.
  • Do you have measurements that take into account program complexity or programs with interdependencies?
  • How is usability measured? Is development effort success tied back to the systems specifications, and does the sponsoring department sign off on that document? To take usability stats a step further, do measurements distinguish between responses from novice and expert users? Maybe measurements should be based on such things as percentage of features used by your clients after 30 days, the speed with which users of the developed product can complete a task, and the number of operations normally performed that cannot be done with your system.
  • How does the QA process factor into developer metrics? Is there any tracking of bugs that gets past the QA folks?

Throw in a dash of project measurements
When development projects are run with formal project management, the project managers usually measure the number of tasks and projects completed on time and within budget. These statistics are a great way to gain a sense of one’s ability to produce when combined with other development-specific metrics. However, they don’t assure that the developed application will actually function. If project measurements appear to have more overall weight than development-specific ones, you may need to push for change. And don’t let an emphasis on project time constraints erode your good coding habits. You could end up completing all tasks on time, but you may not have time to properly comment your code. Or you could end up with a buggy application.

Pull it all together
A combination of project on-time, on-budget measurements, code quality, output metrics, and application usage stats should be used to obtain a complete picture of your performance. Efforts to gather and use isolated stats will not provide an accurate account of you or your team’s abilities and ultimately won't prove useful to management. Take a close look at what’s being measured of you and your applications and work with management to create measurement techniques that add value to your organization.

Editor's Picks

Free Newsletters, In your Inbox