Accuracy errors in project estimates can send IT budgets into disarray, and possibly threaten your own performance and position. And while project management (PM) estimates are just that—estimates—there are several approaches to getting a clear idea whether they’re on target.
There are many different models and approaches CIOs and tech leaders can use to effectively gauge the accuracy of project management estimates. There are also best practices that can provide a good comfort level as to how well the project team is providing cost estimates and forecasts.
The value of historical project data
A key factor in good estimate accuracy is updating estimates through the entire system development life cycle. As you get closer to project completion, most unknowns are better understood, and estimates become more likely predictors of actual results. CIOs also should obtain several estimates—optimistic, most likely, and pessimistic—to gain a sense of the potential project time and cost range.
If you have empirical data on similar completed projects, you already have some initial information to gauge the accuracy of PM estimates.
A project manager needs to use a good tool that will allow him or her to create an initial estimate of the project’s physical and human asset costs and schedule, and this should be placed into a baseline in the project plan.
A baseline is a project’s initial budget. End results of all previous projects can then be compared to project baselines to ascertain the accuracy of PM cost estimates and forecasts.
This metric can be a very good gauge of how accurate your current and future PM estimates will be.
Best practices for PM estimating
If you don’t have previous data on project performance, there are some best practices that can lead to good cost estimates. Keep in mind that even if you do have historical data, it’s not a good idea to use that data alone. As I have found in the past, empirical data is not always a good predictor, but it can be useful for initial estimates.
Employ modeling techniques at project start
Best practices in PM estimation call for accurate predictions of a project’s time and costs. To begin this process, you must know what tasks or deliverables are expected in the project. After you know the tasks, then estimates can be made as to timeframes and associated costs.
The use of modeling techniques in the requirements-gathering phase of a project is the best method I’ve seen to drive out tasks. If your project team is creating context models, event models, and information models—such as entity relationship diagrams (ERDs)—your project team will likely uncover all end user requirements and be able to develop a viable task list.
Context model: A good context model defines all of the project’s activities and processes. It involves data flow diagrams, as well as internal and external inputs and outputs. It will show the entire scope of the project system and will greatly reduce the potential for scope creep. Minimizing scope creep will allow your initial estimates to better equal final project results.
Event model: The event model details the requirements of a project from an end user’s perspective, rather than internal system events. An event list is created describing all customer or end user interactions with the new project or system. Event modeling is quite simple to perform, and will further ensure that your project team leaves no stone unturned. This also will provide a more thorough project task list or work breakdown structure (WBS).
Information model: The information model, which is graphically displayed by ERDs, shows all of the data that needs to be captured by the project system. This is also known as a data model. Capturing all of your data requirements will enable you to double-check the event model. The information model is probably the most important of the three because it ensures that all project tasks become known.
Utilizing these three simple modeling techniques results in a thorough listing of project tasks, which in turn provides a complete WBS. When the WBS is placed into a good PM tool, accurate estimates and forecasts for any given project can begin.
Insight on modeling techniques
David A. Ruble’s book, “Practical Analysis and Design for Client/Server and GUI Systems,” provides excellent detail on these modeling techniques. If you need to gauge how well your project team is providing cost estimates, ask if it is performing the three simple but effective modeling techniques that I outlined above.
In my experience as a project manager, modeling is not often done. The estimation process is usually haphazard and not well thought out. Actual project results almost always deviate from estimates unless a good modeling process is employed. A good modeling process creates an accurate WBS to base estimates on.
Using these modeling techniques will provide your project team with a good start in the planning process, and will ultimately create viable project plans. A good project WBS will provide the basics for viable project estimates and forecasts.
Top-down, bottom-up, and parametric modeling
Once the WBS is known, the PM team needs to provide estimates for time and cost. The best time and cost processes I’ve found are generally referred to as the top-down approach, the bottom-up approach, and parametric modeling. These estimation models enable a project team to more accurately predict and forecast task costs and timeframes from the good WBS created by the best practices of task listing noted above.
Top-down models: This generally involves the judgments and experiences of senior management based upon previous data and experience. It is then sent down to lower layers of management and staff to further define project costs. Small but important tasks in the WBS can be negated with this model, although it has been shown to be a good initial estimator of overall project costs at times.
Bottom-up modeling: As indicated, this is the reverse of the top-down approach. It begins at the lowest layers of enterprise staff, who offer insight into project costs initially, and then the model is sent up the management chain for refinement and changes. Bottom-up techniques tend to include most all of the costs that a project undertakes, and often create a more thorough estimate. Bottom-up models generally are regarded as more thorough and accurate than top-down models. One criticism of this method is that lower staff members can try to "empire build" through this model by adding too many costs and overstating individual importance.
Parametric modeling: This approach uses mathematical models, such as the construction cost model (COCOMO). COCOMO utilizes lines of code (LOC) methodologies or function point analysis in software-related projects to ascertain project costs. Parametric models can lead to some very accurate results in providing project cost estimates and forecasts.
The issues associated with this approach are that parametric models may involve arbitrary assessments of costs that are not always realistic, and may in fact distort project estimates and forecasts. They also tend to be a little more difficult to understand and more time consuming to produce. Nevertheless, if your goal is to deliver truly good estimates and forecasts of a project’s time and cost, and you want to spend the time and money on developing parametric models, then they can be very good estimators.
What’s the best process?
It has been my experience that the best process and procedure in delivering good project estimates and forecasts is a combination of the methods. I have found the best results and most ease-of-use in project estimating from using the context/event/information model process for creating a great task list.
Once projects tasks are known, I generally use a bottom-up approach, where lower level players provide general estimates of time to the tasks. I usually ask for how many days or hours they expect a task to take. I then apply a 10- to 15-percent markup on these estimates.
I also ensure that I keep data on all project player estimates, and create a factor for each individual that addresses if they typically under, over, or accurately estimate their time efforts. Once this factor is applied, I then have good estimates for time per task. Knowing general labor costs and applying a good project management tool can easily derive estimates for overall project cost and time.
If an estimate for a project is needed before these best practices can be employed, I recommend the top-down approach that looks at empirical estimates from previous projects by high-level players.
I also must emphatically advise that an estimate is just that—an estimate. It is a statistical probability, and nothing else. Initial estimates without going through the best practices noted above will have a great degree of risk. The best method is using the three-model approach in task creation, along with the bottom-up method of scheduling, and entering that data into a good PM tool with cost parameters built in.
Nevertheless, CIOs should realize, and accept, that project estimating and forecasting is an on-going process, since projections improve as you move towards completion. Lastly, you should always compare end results to initial baselines to see if your model is working well for your particular organization.