Tech & Work

Understand the cost of collecting metrics

Collecting meaningful data will help you make business decisions. But when is it too costly to collect this data? Project mentor Tom Mochal offers these tips to help you make the decision.


Each week, project management veteran Tom Mochal provides valuable advice about how to plan and manage projects. Tom first describes a common problem scenario, based on real-life situations. He then describes a solution, using practical project management practices and techniques.

Collecting baseline data on a project is necessary if you want to accurately measure success. But how do you convince people that it is worth their time and effort? And is the expense of collecting the data justified? Here is how one project manager solved the problem.

The dilemma
James was nearing the middle of his project to install new document management software for the Legal Department. Part of the business justification for installing the software was the expected increase in productivity. To measure an increase in productivity requires a set of baseline metrics that show how long it takes to store and retrieve documents today to compare with data collected after the new software is installed.

James offered his account of the dilemma. “We built time into our workplan to collect these productivity metrics, but we didn’t take into account the effort the business users would need to spend. Our users are complaining about having to measure and track this information. They are busy. They don’t have time to be collecting a lot of extra information.”

“What information are you trying to collect?” I asked.

James explained that he gave users a template for tracking how long it currently takes to find historical documents, but he encountered a problem when users complained that they didn’t have time to document their work.

“Well, maybe they have a point. Everything we do on a project, including the collection of metrics, needs to make sense from a cost/benefit perspective,” I said. “How much time does it take to fill out the form?”

“Some of the people are saying it takes a lot of time. But we’re not really sure right now,” James replied.

“You need to start by understanding the time and cost of collecting the metrics,” I told James. “Sometimes people don’t like to do new things based on emotion and a resistance to change. However, sometimes the pushback is based on facts. Your team will need to dig deeper to understand the effort associated with the collecting of this metric and work with your sponsor to make sure the value received is worth this effort.”

Mentor advice
Although most business decisions should be based on meaningful data, there is a cost to collecting metrics. A problem occurs when the time and cost to collect the data is greater than the value of the information you gather.

The best metrics are those that can easily be generated by the computer. For metrics that require manual effort to collect, there are three questions to ask:
  1. How much time and expense does it require to gather the information?
  2. Are there less expensive, alternate metrics that will approximate the same result?
  3. Is the value of the knowledge gained worth the cost? As usual, value is in the opinion of the person paying for the information—usually the sponsor.

In James’ case, he needs to talk to the users and estimate the time it takes to collect the metrics. Then he should look for alternatives.

For instance, if the users collect the information for one hour per day, or one day per week, can the overall time frame be extrapolated from there?

James also needs to take the alternatives and costs to his sponsor for resolution. The sponsor may be willing to have people collect very accurate data because of the resulting business value. Or, the sponsor may decide that decreased accuracy, along with substantially reduced collection costs, may be the way to go.

Project management veteran Tom Mochal is director of internal development at a software company in Atlanta. Most recently, he worked for the Coca-Cola Company, where he was responsible for deploying, training, and coaching project management and life-cycle skills to the IS division. He's also worked for Eastman Kodak and Cap Gemini America and has developed a project management methodology called TenStep.

Tell us about your project management problems. Tom Mochal will answer those that affect the widest number of readers. Post a comment below or send us a note.

Editor's Picks

Free Newsletters, In your Inbox