Project Management

Unrealistic project estimates: The IT consultant as an enabler

All IT consultants have given clients unrealistic estimates at one time or another. Get tips on how to resist the pressure to give ad hoc estimates.

Here's an all too familiar scenario:

Bob the client calls and says, "We need to add this small feature to our application. How long will it take?" You respond that you need to draw up some requirements and talk to users, but Bob counters, "Oh, just gimme a ballpark estimate."

The gears start turning in your brain as you imagine implementing the feature as described. Your natural optimism thinks you could complete the work in three days, but you know you can be overly optimistic, so you answer, "Two weeks."

Guess what? You're still short. You forgot these important factors:

  • Requirements. Your client contact, whether they're the CEO, CIO, project manager, or whomever, cannot have a complete picture unless she or he is the only user -- and even then it's dicey. Besides, there's no guarantee that what you took from their brief description even matches what they meant.
  • Design. You don't want to jump in and start cowboy coding without a proper design of each component. I'm not talking about flowcharting everything or generating piles of UML that nobody will ever read again; your design needs to be flexible but well thought out. Even though this seems like it adds time to the project, failing to do this step adds even more time in the long run.
  • Testing. I'm a big believer in writing your tests before you write the code to satisfy them -- or at least, in parallel. This is another step that saves you time down the road.
  • Iterations. No matter how well you define the requirements up-front, somebody will forget something important or something nobody even knew. You need to build in time for providing prototype versions so the client can use the product and provide feedback that drives new requirements.
  • Documentation. A system that isn't documented will be reinvented. If you want the project to have any shelf life, you need to provide usage instructions for people you may never meet and put enough comments in the code that you'd be able to understand it even if you hadn't written it.
  • Training. No matter how well you write, users may not be able to grok what's going on without some personal guidance.

Taking shortcuts that cost more time in the long run

Consultants often leave out some or all of the above steps in an attempt to provide results quickly. Bob may be impressed when he gets the new feature in less than a week, but pretty soon, he'll find out that the project isn't quite done yet. In fact, you've only completed the first iteration, and even that was half-baked. The project will probably drag on for months or even years, as you tweak the features, fix the bugs, and give the users the same instructions over and over again.

So, why do consultants take shortcuts that make the journey longer? Here are four reasons:

  • Client pressure. Bob wants you to give him a low number, so he can justify the work. You don't want to disappoint Bob, and you want him to approve the project.
  • Competition. If you aren't the only bidder on the project, you want to be the lowest bidder so you get the gig.
  • Hubris. No matter how much you pad an ad hoc estimate, it still isn't worth the air that it's written on. But once you've stated a number, your pride and your standing with the client won't let you admit that you were wrong. So you cut corners.
  • Pure evil. Some consultants get the client on the hook by intentionally engineering the project to require follow-up work after it's "done." Those consultants give the rest of us a bad name. (They should really read my column, "An independent consultant's code of ethics.")

Fending off pressures to give ad hoc estimates

I have three recommendations for how to resist these pressures:

First, consultants have to be aware that these pressures exist. When I was a newbie consultant, I had no idea why projects always took longer than expected -- my estimate always seemed reasonable to me at the beginning. Learn from past projects -- what went well and what didn't, both in your experience and in the experiences of others.

Second, consultants have to educate clients on what a well-executed project requires. Clients need to understand that by taking a month instead of a week, they'll get a finished product that meets their needs instead of one that does exactly what they said but not what they meant. And even if it does require modifications after delivery, they'll be easier to implement without breaking something else.

Third, consultants should not give ad hoc estimates. If your client won't stop bugging you until they get one, then say, "sometime between a month and 10 years" because frankly, that's about as precise as you can be without more information. Instead, you should push for approval of an exploratory project to establish requirements and compute a decent estimate.

Building a reliable reputation

Unfortunately, because so many self-styled consultants are willing to cut corners, they've set the expectations in clients' minds about what estimates mean. The more intelligent clients have come to expect that the project estimate will be far less than what's really required when all is said and done. So when they receive a more accurate estimate, they're likely to think that the actual timeframe for having something usable will be much longer. You need to build a track record of reliability before your client can trust your prognostications.

Get weekly consulting tips in your inbox TechRepublic's IT Consultant newsletter, delivered each Monday, offers tips on how to attract customers, build your business, and increase your technical skills in order to get the job done. Automatically sign up today!

About

Chip Camden has been programming since 1978, and he's still not done. An independent consultant since 1991, Chip specializes in software development tools, languages, and migration to new technology. Besides writing for TechRepublic's IT Consultant b...

44 comments
biancaluna
biancaluna

These are all issues to keep in mind and consider when responding to and preparing estimates. Also, I try to use order of magnitude statements - at the conceptual stage + / - 75%, at the planning stage 50% and then a tolerance of whatever % is realistic in view of my confidence in the ability to deliver on time and on budget. I try to apply this mehtod consistently to every stage in a project, but that is not easy to get through sometimes. What I have found is that the organisational culture is a big money pit. I've dealt with clients who take 3 months to make a very simple decision purely as nobody has the authorisation to do anything or is too scared to make a decision. Or rubber band the scope such that it oinks out of shape at every opportunity. There are so many methods, it is a bit of a blackish art though. Yeah, we can get MS project to run a Monte Carlo analysis on your estimates, but if I know a client and I know their culture and I actually learn from the past, that drives my estimates more than hard metrics.

Ed Woychowsky
Ed Woychowsky

Step 1: Ask the people doing the work for an estimate. Step 2: Management divides the number by three. Step 3: Punish the people doing the work for taking three times as long to complete the project as outlined in step three. In order to make project estimates more accurate the following was added: Step 1B: Multiply the number by three. ;)

robin
robin

Underestimating for all the cited reasons is epidemic in the larger pool of developers from which consultants come. In my experience, consultants (including consulting firms, not just individuals) are more likely to learn how to estimate responsibly, which is one of the reasons clients choose to work with us over their internal staff, which seldom seem inclined to make estimates that come true--except that they will fail to deliver quality in the estimated time and budget. Like reisen55, I have long lived by my estimates, honored them, and gotten much better at hitting the mark. And, yes, I still infuriate my wife when she's still waiting after I've told her I'll be done with something in a few minutes.

jeff
jeff

Simple concept: If you don't understand the scope fully, you can't quote. "Oh, just give me an estimate" is a trap you need to stay out of at all costs. Jeff Yablon President & CEO Virtual VIP Computer Consulting, Business Coaching and Virtual Assistant Services

reisen55
reisen55

I was given a referral by a good client for a small medical house that had a malware problem. Visited the office and discovered ANTIVIRUS 2009 which is Norton look-alike-clone that is easy to remove through control panel. Did so, said it was fixed, charged one hour of time and walked out the door. No sooner was I 20 steps out did the receptionist come screaming back telling me it was back again. So I returned and this little monster is invasive as hell. Took over web pages, Google displayed registration boxes = horrible. But, they paid me to remove it so I grabbed the box, and discovered a wonderful little utility - MALWAREBYTES ANTIMALWARE which did the job. Took another 3 hours to do that and backup their data but.... I said an hour, was paid for an hour and for a new client I honored that commitment. Good client too and I periodically stop by to check on issues. If you give a bad estimate, lump it if you can and keep the client, recognize your error, document solutions and be smarter the next time around.

Joe987654321
Joe987654321

This article is naive. I have been through all the different approaches to providing estimates and have struggled to educate people about the worthlesness of ballpark quotes. In the end I have come to the conclusion that meaningless ad-hoc estimates are the only kind of estimates you can provide. The fact is that unless you're dealing with a very big, mature or well established customer, people just want simple easy answers and will always make cost based decisions on superficial information. If you don't provide an easy answer someone else will and you won't get the work, it's that basic. It's far easier to limit your deliverables to meet the time frame, and then get the client to make allowances for additional time to give them what the really want than to try to give them what they want in the first instance. This isn't the fault of the consultant, it's the fault of greedy stupid customers. Sadly, this is also why bespoke software is universally rubbish and why I avoid consulting work in favor of shrinkwrap product development when I can.

Tony Hopkinson
Tony Hopkinson

in my current job, usually based on an appalling lack of information. I tend to do it based on my estimation of risk. Which is based on the lack of concrete requirement. When I was consulting, I was brought in as a either an extra general resource, or as a one off specialist skill set. Estimates are really how many people you need to get this dome by an external deadline, or how much will it cost to schieve. As such I don't have a problem with giving ball park estimates, but like you I generally make them useless, I've been know to say five minutes to five years, dependant on risk factors. The psuedo science of estimating is somethng I've always considered a bad joke. Those the estimate should be +-10.675 % of actual guys kill me.

kevaburg
kevaburg

was a nightmare! It should have been really easy! The client asked me if I could help him optimise his database to allow for another office in another town and some additional employees that would work there. We spoke on the phone and most of what was being said made sense. He sent me through a .csv file with the data and I got to work. His is only a small business and we spoke about using MS Access as his solution and he was all for it and within a small amount of time, I had built his database and everything was ready. I got to his office to find him working away so I set everything up ready for the demonstration of my offering. His first words were: "What is that? My database looks totally different." And it did. His "database" was an Excel spreadsheet and the "security" he wanted wasn't user accounts, rather the change tracking feature that Excel offers. I told him we spoke about using Access and his answer? "Well, I meant Excel!". Great. Fortunately I understand enough about Excel and his requirements to put his solution together "on-the-fly" (I'll get flamed for that decision for sure!) and it worked. Then it had to be usable across his network which was wireless at his location. Excels workgroup sharing computing was enabled and once again I got to work. The workstations on the network were privately owned laptops, some with AV, some not. The router supported WPA2, one of the laptops supported only WEP. His building was nothing better than a Faraday Cage and my ad-hoc site survey showed the worst reception I have seen for such a small building. Needless to say, this was my biggest failure as a consultant. I should have gone there 100% (or as close I could be) prepared. The site survey and hardware inventories should have been completed well beforehand and a detailed discussion of his requirements and looking at his current system would have offered me an insight that was essential to the project. Since this project, I have never done anything that I have only been half-prepared for. If I am technically or logistically not able to carry out a project, I tell my client that, we shake hands and I leave. Soemtimes they come back, sometimes not. But from that point, I always let the client know that their requirements are my no.1 concern and if I could not competently do what was sked of me, then I would pass. I have a good name as a consultant now and the only way to keep that is to be up-front, honest and extremely well-prepared. Thanks for the article!

HAL 9000
HAL 9000

The bottom line is any Estimate is at best your worst guess. Personally I don't give estimates on anything I will give a Quote to build a new computer but everything else is just a Guesstimate. Even the printed forms that I may send out when really pushed say Guesstimate not Estimate. I figured out a very long time ago that estimating anything was a recipe for disaster and caused me so much bad Blood and friction that I figured out that it was easier not to play that game. Now when really pushed I will say something like [b]It will be ready sometime in the next 25 years.[/b] That is about all it is possible to really say. Also Chip I have to point out the error in your statement here [i]Third, consultants should not give ad hoc estimates. If your client won?t stop bugging you until they get one, then say, [b]?sometime between a month and 10 years?[/b] because frankly, that?s about as precise as you can be without more information. Instead, you should push for approval of an exploratory project to establish requirements and compute a decent estimate.[/i] Giving the month bit is so over optimistic that it's downright dangerous. The only time that I will ever say that I can have something out within a month is when I'm talking about one of my bills to a client and even then it's not always possible to keep on top of the paperwork so it may take a bit longer to get around to actually Printing out the bill. I'll not even mention anything about posting it when this is required that can delay things even further. Now that I no longer do things like this I have found that I don't get the requests for immediate Guesses they ask me is this possible and will it take [b]Forever.[/b] ;) I have found the best way to please a client is to tell them the truth. Initially they may not like what they hear but I have yet to have one not come back and want the work done. The closet thing that I will give is a ETA on arriving on a site to do some work and even then my current crop of clients don't expect me to be on time they expect me to arrive within several days of that ETA depending on what other problems they have. I now get things like [i]If you are passing this way in the next month or so can you drop in?[/b] for Routine Maintenance, even the urgent work where lots of money is being lost per minute I get requests like [b]Can you try to get here this week?[/b] Mind you I'm not out chasing work and I don't need to set unrealistic goals to attempt to get the work. So maybe I'm the wrong person to post here. :D Col

santeewelding
santeewelding

Re, your recommendations for how to "resist" pressures. They come to you, don't they? I know they come to me. The stage is set: You are in charge of pressure.

Sterling chip Camden
Sterling chip Camden

Yep, the best that metrics can do for you is flag some dependency you completely overlooked or discounted. I like your idea of voicing a greater margin of error in the early phases. That's spot on.

Sterling chip Camden
Sterling chip Camden

In Metamagical Themas, Douglas Hofstadter discusses his rule: "Every project takes longer than you think it will, even if you take Hofstadter's rule into account." He muses over the implication of this rule: if you take Hofstadter's rule into account, then the project will take even longer than it would have if you hadn't. It seems to me that management often operates on that assumption: set the deadline unrealistically, so employees (or consultants) work harder to try to make it, even if they can't.

Sterling chip Camden
Sterling chip Camden

Employees often have less on the line when it comes to making bad estimates. We're forced to become more careful, or die.

Sterling chip Camden
Sterling chip Camden

is "be smarter the next time around". Don't keep making those ad hoc estimates, so you don't keep having to honor them.

Tony Hopkinson
Tony Hopkinson

muisinterpretation there though is there. What if they's come back with oh I thought you were going to rebiuld and secure all our machine ans network, set up an enterprise AV, get AD going and install and configure all the switches to fix it. In that price... It's the sort of (scope :p ) creep that we can be faced with.

ArnoldZiffle
ArnoldZiffle

Sorry this is not the same case as Chip's article. He was talking about an addition, revision or outright original development effort. That is not what you were faced with. May I respectfully suggest that your response to the client probably should have been a range, telling the client, "it usually takes x to y hours for this sort of thing." In other words a Time and Materials job.

Sterling chip Camden
Sterling chip Camden

... and I'll take that as a compliment. What do you say if I call you fatalistic and your prophecies self-fulfilling?

Sterling chip Camden
Sterling chip Camden

Not unlike psychic predictions. And like the latter, they're more accurate when they're more vague.

Sterling chip Camden
Sterling chip Camden

... probably does create a false expectation that it could be done "maybe" in that timeframe. I'll have to eliminate that in the future. Thanks.

Sterling chip Camden
Sterling chip Camden

Sometimes you have to step back and get the right perspective. The "wanting to please" pressure is the one I feel the most.

DesD
DesD

but that's probably where the majority of bodies are in the IT world. Long, long ago, I read of a study by the Rand Corp, who got some people to estimate, then write, a piece of code. The best of them took 4 times longer than their estimate, the worst took 10 times longer! Some time later (6 months to a year, I think) they got the same people to redo the same exercise. Yep, you guessed it. The best of them only took twice as long, and the worst "only" 7 times. Moral: if you're good and you know it, don't double your estimate, quadruple it. And if you're not so good, add a zero!

ArnoldZiffle
ArnoldZiffle

Have you ever had to fire a client because of what I like to call, "Creeping Elegance?" All those little things they think of while you're working on a project. That justifies a detailed design, singed off by the client. And don't budge even for a simple change or addition because once you open the door...

reisen55
reisen55

In thinking about my short tale, I probably sat down, looked at the virus itself and removed it through simple procedures. Got it out and said "An hour" and that closed this issue. Some can be that easy. So in this case I closed the sale and when walking out learned it was a much more invasive, larger issue than first thought.

Marty R. Milette
Marty R. Milette

When I first started consulting, my rule of thumb was to come up with a best guess estimate and double it. Over the course of 20 or so smaller projects -- this worked out close enough most of the time. When working in offshore development (supplier side), and for larger projects -- I found it useful to base estimates of time and effort on benchmarks of similar previously completed projects. Eventually, if you do enough similar types of projects -- you'll get a pretty good idea of exactly how long they'll take and the kinds of pitfalls you can encounter that will cause delays. Of course, the MOSt important thing is to remove problems caused by 'dependencies' on any external party (such as the client!). So any contract MUST specifiy an SLA for client and/or third-party deliverables and a 'standby' fee.

Sterling chip Camden
Sterling chip Camden

I gave a fixed-price quote for setting up a system, loading up the software, and setting up the initial books on Peachtree accounting software. By the time my assistant and I got the books set up, they were out of date already. We ended up on the hook for about three months trying to get to an accurate balance sheet. Since then, I'm hourly.

metalpro2005
metalpro2005

If the client is more mature, it will see the 'quotations' as an invitation for a more thorough conversation. I for one would like to see the reason why this uncertainty is there in the first place! That should be the real incentive to this answer rather than being nailed down to the actual figures? It is interesting to see that I have the same strategy concerning managing expectations as you have.

HAL 9000
HAL 9000

I can cure you of that if you want me to. I'll even enjoy it as it involves a unique form of [b]Aversion Therapy.[/b] Every Time I think you will want please someone I hit you over the head with a Big Hammer. It's so relaxing for the one giving the Therapy. It worked so well on the Goodies that I found it the perfect cure for everything that my staff suffered from. :^0 It even worked so well that I educated the sales people not to tell clients when my department would be doing any work. ;) Col

santeewelding
santeewelding

As evidenced by all your work here, you will grow out of it, eventually. At which time you will be insufferable to lesser souls.

thomas_w_bowman
thomas_w_bowman

My first brush with this was "How many hours to by end of next Month ?". So he was already thinking 320 Hours, when I broke it down I could not see how it could be done in less than about 1600 - plus the work was not suited to split up in more than 3 pieces (5 developers could not finish in 2 Months). I told him that IF THERE WERE NO CHANGES TO REQUIREMENTS it might be done in 4 Months with 3 Developers (Full-Time, because we know that 25% is really 0%). He panicked, having just had another PM quit due to the crazy deadline... So I told him how far we might be in 2 Months (it was a Flexible Benefits project) - we might have enough developed to track contributions, the development to automate paying Insurers would have to be done after the sysem was 'announced' and "Implemented" (the Company Pres announced a 'ready' date without asking IT). We phased the development, and it did cost A LOT more to have manual processing while we were completing development - but we delivered the 'illusion' of having it in on time (the deductions were on paystubs - employees didn't know what accuonting was having to do to manually pay insurances). And, of course - I documented the many Spec changes - which was handy when we were done 6 months later and questins arose as to why it took so much longer than estimate (they had to own their spec changes and their severe impact on development, testing and implementation). Recently I am dealing with a Manager who thought we could test a complex change in 4 hours - then migrate to Prod... I pointed out that the first 2-3 tests took longest, since repairs can be involved - and simply asked if there was any risk to testing in Prod (yes, there was - Millions of $ at risk). I will only give initial estimates for how long it'll take to make a more precise estimate (based largely on Defined Requirements and probability of spec changes - too often the Business is unsure of what they really want. I like to give 3 estimates: 1. 'Raw Meat' coding, perhaps some reports to help manual process, 2. 'Phase I' to build on but offer some automation savings quickly, and 3. 'Pie in the Sky' where the System will do nearly everything (generally not practical - unless done in Phases).

ArnoldZiffle
ArnoldZiffle

Although I've not read the Rand study I have taken to doubling my estimate. I also make sure the client understands that an estimate of hours is not, linear.

santeewelding
santeewelding

There are some things I am good at, and I know it. Nevertheless, I approach those things the way I assign less than one-hundred percent probability to the sun rising tomorrow. I clash here with those who don't.

Sterling chip Camden
Sterling chip Camden

... but not due to scope creep. I've almost always billed by the hour, so scope creep was never a problem for me.

Sterling chip Camden
Sterling chip Camden

Scope and prior relevant experience. The larger the project, the more room for the unexpected. The more unfamiliar the territory, the less you can be certain about. One of the biggest mistakes in estimating is the idea that the only factor is the overall size of what needs to be done, and that the effect of that size is linear. The amount of work has closer to an exponential effect, and the unknowns are much more important than most people realize.

Sterling chip Camden
Sterling chip Camden

Yes, the first thing to try to establish is the degree of uncertainty. Unfortunately, uncertainty often masks other uncertainties of which you can have no knowledge.

Sterling chip Camden
Sterling chip Camden

I'm working on a regex parser right now, so I'm keenly aware that just searching for /ol/ involves at least two state transitions. That's a candidate for two phases.

Tony Hopkinson
Tony Hopkinson

If it's bigger than fixing Helol Wolrd", there's at least one worthwhile phase in there.

Sterling chip Camden
Sterling chip Camden

I hadn't heard the term "raw meat" used before -- that's pretty good. I'm a big fan of phased approaches -- the more phases the better, provided you don't have too much overhead involved in moving from one phase to another. When you break down a project into phases, you tend to eliminate dependencies between phases and you can order the phases by priority and drop the ones you don't have time for.

Editor's Picks