At my house, before we send the kids to bed, we try to read to them. Right now, we’re working our way through a book that was one of my favorites while growing up: Robert Heinlein’s Have Space Suit—Will Travel. Toward the end of the novel, the young protagonist discovers that his father, whom the kid had always considered a bit of a plodder, was some kind of genius. (This provided my own children no small measure of mirth.) As proof, the son learns that his dad had long ago written a classic mathematical text, entitled On the Statistical Interpretation of Imperfect Data.

Over the years since first reading that novel, I’ve wondered about that fictitious mathematics text—why would anyone care about interpreting imperfect data? However, as an IT manager, I’m guessing you often find yourself having to make decisions with less-than-perfect information.

Discuss this article for a chance to win a TechRepublic coffee mug

In our Discussion Center, we’re talking about how to best use the data you have. To add to this discussion, post your comment to this article. Each week, the person who provides the best feedback to an Artner’s Law column will win a free TechRepublic coffee mug.

To find out who each week’s winner is and to subscribe to Artner’s Law, sign up for the TechRepublic TechMail now.

In last week’s column, “Think before you measure,” we talked about the importance of choosing the right metrics, especially when measuring employee performance. This week, we’re going to look at how to approach decision making when you do have some data—but not all the data that you would like.

Why we rarely get all the data we need
Not too long ago, I wrote a column (“Knowing when speed matters”) that discussed the need for speed in application deployment, arguing that rapid deployment was often more important than delivering 100 percent of the application’s feature set. In other words, there are times when it’s better to be fast than to be perfect.

Unfortunately, speed comes at a price. One price you pay for having to move quickly is that you often sacrifice the ability to gather enough information before having to make a decision. You hope your systems are flexible enough that you can change directions later, if further information indicates you made a bad call the first time.

In addition to speed, another problem facing technical managers is a glut of data. As we discussed last week, it’s important to choose the right metrics. Unfortunately, some processes are so complicated and produce so much data that just wading through the information is impossible.

Another common problem is that by the time your systems can analyze the data, it’s out of date. Most of us are familiar with the just-in-time inventory control systems made famous by Japanese firms in the 1980s as a method of reducing inventory and resultant costs. For these systems to work, warehouse managers have to have instant access to critical information on order demand and inventory supply.

Sad to say, most technical managers have nothing like that level of reporting available for most of the projects and processes they have to oversee.

What to do about it
Whatever the reason, if you’re like most of the IT managers I know, you often find yourself having to make decisions on the fly without all the data you’d like. In that kind of situation, what principles can you use to guide your decision making? Here are some general rules to follow:

  • If a number looks wrong, maybe it is: You’d think that with all we know about computers, as technical managers we’d be more suspicious of data that looks wrong. All too often, we assume the number is right and we’re wrong. When I was in high school, I remember my dad railing about some guy who worked for him issuing a quarterly sales report that estimated net profit for the quarter (my dad worked at the Major Appliance division of General Electric) at $4 million, instead of $40 million—a difference of an entire order of magnitude. When my dad confronted the guy about submitting a report that listed net profit at only 10 percent of budget, he responded that, yes, the number looked fishy, but that’s what the finance department had given him. As you’d expect, when they dug into it, it turns out that someone had misplaced a decimal.
  • Try to put individual data points in context: It’s very difficult to look at a single point of data and draw proper conclusions from it. For example, suppose the CIO of your organization came up to you one day and said, “You know, I think you’ve got a problem with Jenkins. He’s been two hours late three times this week. Does he have some kind of attitude problem?” Before responding, wouldn’t you want to know if Jenkins had been working overtime, preparing the new Active Directory implementation?

    To take another example, suppose a project manager mentions to you that one of your programmers consistently missed more milestones than the rest of the programmers. Before deciding that he or she had serious performance issues, wouldn’t it be prudent to look at the kind of development projects the programmer had been assigned? Wouldn’t it also make sense to ascertain who set the milestone deadlines in the first place?

  • Hard data can lead to bad decisions: Lawyers always say that hard cases make bad law. The same thing can be true for technical managers. For example, suppose you’re responsible for data integrity at your company’s four data centers. Let’s also assume that there is a server failure at one location, which is compounded by the fact that proper backup procedures weren’t followed.

    When trying to ensure that such an event doesn’t happen again, you could concentrate on any data points you get from the data center where the problem occurred, identify the problem, and try to make sure it doesn’t reoccur. On the other hand, it might be more prudent to look at the one data center that has best practices for data backup and retrieval and compare those best practices with what happened at the problem data center. In other words, instead of fixing what went wrong, maybe you should focus on how to repeat what went right.

Whatever your approach, you’ll all too often find yourself having to make a decision in a hurry without all the data you’d like. If possible, try to get some additional time in order to make a more-informed decision. If you can’t do that, at least walk away from the decision for a moment—to get your ego and emotion out of the way.

Good luck!

Share your retention strategies

What’s your strategy for managing and using the data you collect? How do you keep projects moving when data is lacking? To add to this discussion, post your comment to this article. Each week, the person who provides the best feedback to an Artner’s Law column will win a free TechRepublic coffee mug.