CXO

Weighting a project scorecard's criteria

If your client values completing your project on time more than keeping to a budget, you can overspend and still consider your effort a success, if you weight the criteria in your project scorecard.


A project team rarely wants to put itself into a position in which the opinion of one person can make all the difference on a lengthy, complex project. At the same time, few sponsors probably are willing to give a single “thumbs up or thumbs down” response. There are usually aspects of the project that the sponsor is happy about, and others that the sponsor wishes went better.

All this begs the need to create a balanced scorecard with multiple success criteria. However, not all success criteria are created equally. Let’s examine the practice of weighting your project success criteria.

Second in a series
This is the second half of a two-part series that looks at developing a project scorecard to better judge the success of your projects. Last week’s article included basic examples and featured examples from Tom Mochal’s experience.

The weighting system
As you consider multiple factors to determine project success, the last area you need to address is a weighting system. This will be necessary because not all of the performance factors are of equal importance.

For example, completing your project by a certain date may be more important than whether you hit your budget target. In fact, you may have to overspend to make sure you hit the deadline. So, on that project, meeting the deadline date should be weighted more heavily than meeting the budget target.

The way you establish an overall weighting formula is to start with a maximum point total and divide the achievement of that maximum total into the number of success criteria. Let’s look at an example of a project with five success criteria.

The achievement of all the success criteria results in a 100-point maximum. In other words, if the project team did everything perfectly, its total score would be 100 points. Let's also say that the five criteria are deadline, budget, quality, client satisfaction, and sponsor rating of success. You would develop a scorecard grid like the one in Figure A.
Figure A
# Scorecard criteria
Weighting
Results
Score
1 Deadline: Complete by December 31
25
12/31
25
2 Budget: $250,000 plus or minus 10%
10
$270,000
10
3 Quality: Five question survey, averages at least 3.5 out of 5
20
3.2
10
4 Client satisfaction: Five question survey, averages at least 3.5 out of 5
20
3.4
15
5 Sponsor rating: Three question survey, averages at least 3.5 out of 5
25
4
25
6 Total
100
 
85

You can get very sophisticated with this scorecard system. However, you’ll find that each attempt to get more fact-based, and less subjective, requires you to get more and more precise in defining the overall scorecard rules. For example, the following must be taken into account as you get more sophisticated.

Define what the numerical total represents
On the project above, you could say that any final rating greater than 80 represents success. In that case, the project above was successful. If there was a bonus to be paid for project success, you could say that a full bonus would be paid for an overall rating of 80 or greater. No bonus (or a partial bonus) would be paid for a score lower than 80.

Define whether partial credit goes to scores that don’t meet the criteria
In our example, the project went over budget by $20,000. This was within the acceptable range of plus or minus 10 percent ($225,000 to $275,000), so the team received full credit of 10 points.

Partial credit
On the other hand, the overall quality rating was 3.2 and the target was 3.5. The question is whether the project team receives 0 points since it missed the criterion or receives partial credit for getting close.

In this example, the team received 10 points, or half the possible total. You could imagine that at a lower score, say below 3.0, the team would have received 0 points. If the team had missed that criterion, its total points for the project would have fallen to 75.

If the target were 80, the project would have been viewed as a failure, even though the rest of the criteria were hit successfully, including an overall 4 out of 5 rating from the sponsor. That shows why it’s important to have proper weighting for your project. If you’re going to have this level of algorithmic objectivity, it’s crucial that the team really focus on the important scorecard success factors. This is a powerful incentive.

Decide whether the weightings represent the maximum allowed, or whether the team can score higher
On your sponsor survey criteria, perhaps you could establish a rule that says you receive an extra 10 points for every .5 over your target. In our example above, since the final score was 4.0, the team would have received 35 points for this criterion, which represents the target 25 points, plus an extra 10 points for exceeding the target. Again, if you allow extra credit, you must determine what it means. For instance, if you were paying a bonus on successful completion of the project, going over 100 might mean that the team receives a higher bonus, say 110 percent of the target bonus.

The success formula must be known ahead of time
The key to these rating systems is defining them ahead of time so that the entire team can focus on the areas that are important. If you don't finalize the criteria until the project has already started, you may find out that you’re too far behind in certain categories to achieve them.

If the team feels that it doesn’t have a chance to achieve the targets, the scorecard will result in a lot of negative feeling and poor morale, which is the opposite of what you’re trying to achieve.

The other positive result of building the scorecard early is that it gives the project team time to adjust if there are problems. For example, in the project above, let's assume that the project duration is around nine months. The project manager may send out the quality survey and the client satisfaction survey after three months and after six months. That way, if the survey results aren’t coming as expected, the project team can determine the cause, and the entire team has time to correct the problems before the project is completed.

This is part of a project improvement process. If the scorecard isn’t completed early and the team doesn’t have time for interim surveys, the first time it sees survey results is when the project is completed, which is too late to make any corrections.

Make sure you have client agreement as well
The project team and the client must agree on the overall scorecard ahead of time. You don’t want to be in a position in which the project team is declaring success at the end of a project, but the client feels the project was unsuccessful. If you have agreement on the scorecard, both the project team and the business client should have common expectations in terms of overall project success.

It’s not as easy as it looks
In high school, a final grade is usually determined by a formula based on things like test scores, quiz scores, homework, and attendance. For the most part, everything is number-based, and can be calculated based on the raw scores—pretty simple and no surprises.

On a project, however, it’s not that simple; you have to consider client satisfaction and quality, both of which get into subjective data. Your project scorecard can try to compensate for the subjectivity and attempt to make the process as objective and mathematically driven as possible, just like your high school grade.

However, the more objective and fact-driven you attempt to get, the more complex and intricate your scorecard data collection and interpretation must be. Your scorecard criteria should be balanced and comprehensive. You should have targets that are challenging but reasonable. You also need to weight the various aspects of the scorecard to make sure that the relative importance is established. If you don’t think through these implications, you may have confusion and disappointment at the end of the project when your scorecard results are subject to multiple interpretations.

Editor's Picks

Free Newsletters, In your Inbox