Leadership optimize

Plan your benchmarking approach before starting to collect metrics

When you dive into a benchmarking project, the steps you take to ensure you get the necessary feedback are essential. For instance, not everyone needs an ongoing study of network performance metrics. This article shows you where to focus your efforts.

Let’s say you're an IT manager with several years of management experience in your organization. Your CEO comes to you and asks about the productivity of your IT staff. You reply that you think your staff is productive and your internal metrics all show improvement. However, your CEO has an idea—why don’t you benchmark your staff against those in similar companies to see how they stack up?

Where do you start? First, you need to do some preparation. You need to frame out what you want to accomplish through benchmarking and the type of benchmarking program you want to set up. Let's look at the steps required to do this.

In case you missed it
This is the second article in a series on benchmarking. If you missed the first one, you can check it out now: "Use benchmarking to chart your organization’s performance."

Goal setting
The IT organization covers a wide variety of disciplines. For instance, you can benchmark your network, help desk, development area, telecommunications, and so on. Before you start, then, you need to determine the scope of the benchmarking effort. However, if you ask someone to define the scope, the first thing he or she is going to ask is, “What are you trying to accomplish?” So really the place to start is to define your goals—high-level statements that describe what you hope to achieve. The goal of benchmarking is not to gather the metrics and perform the benchmarking study. Benchmarking is only a means to an end. Your goal might be to become more efficient in your development process or to increase your telecommunications capability to support the business. Once you have some goal statements, you can start to narrow down the benchmarking initiative.

Determine benchmarking model
After defining the overall goals of the study, you should determine the type of benchmarking you want to perform. You have the option to develop and run the benchmarking effort yourself or use an outside company. I've seen it work both ways. When I worked for a major beverage company, I participated in a benchmarking study on development processes run by an outside company. I also participated in a targeted benchmarking program that included only two companies, which we set up and run ourselves. If you use an outside company, it typically has preexisting processes, metrics, and benchmarks that can be used entirely or as a starting point for customization. If you run the program yourself, you may have to develop all of the benchmarking content yourself.

Another area to consider is whether you're looking for a one-time study or whether the benchmarking will be ongoing. Sometimes benchmarking is only meant to take a snapshot of where you are vs. where other companies are. Other times, you may want to benchmark on an ongoing basis, with metrics captured monthly or quarterly. This is more work, of course, but it does allow you to make improvements and compare yourself to the benchmarking group over time.

Comparison subjects
Another important consideration is to determine who you want to involve in the study. If you're a multibillion dollar company, you don’t want to compare networking capabilities with a collection of million dollar companies. While you may turn out to be the best in the study, the information will not help you if you're competing with companies that are larger in size and market reach.

You should determine the industry focus of the companies that you'd like to include in the study. For example, I spoke to a person from a company in the oil industry who told me that the company only wanted to compare itself against other companies in the oil industry. On the other hand, when I worked for the large beverage company, we didn't have that luxury because two companies dominate the industry, and we had a policy not to participate in any programs that included our competitor. In our case, however, we were interested in comparing ourselves against a collection of other large companies in various industries.

Create the detailed model
Let’s look at the information needed for the detailed benchmarking model. If you work backward, you can see that the business benefit to benchmarking comes from improving your processes. You improve your processes by first seeing who is doing better than you and then understanding how they're doing similar work. You determine who is doing better than you by comparing your company to other companies using a common set of benchmarks. The benchmarks are determined using a set of underlying and agreed upon metrics. The metrics are established to provide an indication of how your processes are working. Since all companies use different processes, you must first determine a common set of processes that each company can understand.

Taking benchmarking logic the other way, you must map your own processes into a common detailed process model, then collect metrics for each process and report the metrics to a common collection point. The metrics are combined into numeric benchmarks that allow companies to compare themselves and be ranked. You then look at companies that rank higher than yours to determine what they're doing differently and how you might change to become better.

Now start working on the details
At this point, you've defined your overall goals, benchmarking model, and the types of companies you'd like to include. The direction you take now will depend a lot on the model. If you use an outside company, for instance, you may have enough information to allow it to start to run the process. Hopefully, the outside company will have a detailed technical model that you can use as the basis of the benchmarking study. The outside company can also start to contact target companies to solicit their interest in participating. Remember that the benchmarking program means nothing if you can’t get other companies involved.

For the purposes of this series of articles on benchmarking, I'll assume that you're creating the benchmarking detailed model. In the next article, I'll explain how to define the common processes, metrics, and benchmarks and look at some examples of how they work.