Project Management

Is failure ever the best option for consultants?

Would giving a client a little taste of disaster before your IT service saves the day make the client more appreciative of your work? Read why a TechRepublic member recently contemplated this question.

TechRepublic member Bob Eisenhardt (reisen55) keeps feeding me great topics. This time, Bob relates a case in which doing his job too well led to a client's suspicion of their need for his services.

In a previous article, I related Bob's "I told you so" adventure, in which he luckily happened to be on site on the weekend when a client's hard drive (which he had recommended for replacement) suddenly failed. Bob swapped in a drive that he carried with him and restored from an image he had taken earlier. The client then purchased the replacement drives Bob had recommended, which he swapped in on a Monday night. The client's business didn't miss a beat. A 100% happy ending. That's the way IT service should be conducted, right?

Well, according to Bob, the client made an ostensibly joking comment: "Whenever Bob needs money, gee, these problems happen!" Bob says, "I blew up at the client and told him in no uncertain terms he was damn lucky to have anything back so quickly this morning."

The client then expressed trust in Bob's honesty, but the incident bothered Bob. He discussed it with a colleague, who said, "Gee, maybe if you let the server be dead on Monday morning and restored it that day, the client would have appreciated your work all the more!"

In other words, because the client felt no pain they didn't perceive a real need. For all they knew, the drive never failed at all, because Bob was there at the time and rectified the situation so quickly. All they experienced of the transaction was shelling out the money for the new drives.

I have a couple of problems, though, with the alternatives suggested by this line of thought. First, in order to leave the system down, Bob would have had to adopt one of three stances:

  1. Pretend that he left the system running fine. That's lying to your client. Not only would that be unethical, but as with any lie, no matter how small, you run the risk of getting caught in it.
  2. Tell them that it failed while you were there, but that you needed to get the new drives before you could replace them. In other words, don't volunteer the spare drive you were carrying. That's either another lie, or an indication that you're somewhat callous towards your client's needs.
  3. Tell them that you left the system down because the decision to replace the drives would be up to them. That provokes a "Duh!"

Second, I've always held that we should do the best by our clients, regardless of how it looks. All other things being equal, less downtime is better.

I would suggest instead that you play out the alternate scenarios in a discussion with the client. "If I hadn't happened to be here at the time, or if I didn't carry a spare drive around, you'd have lost a few hours work on Monday. And if I hadn't made that ghost image, you'd be SOL." Perhaps a demonstration would be in order that the hard drive was really shot. Plug it into a spare system and try to boot it, so they can see it's dead.

That's my opinion, but I'm biased towards being truthful. What do the rest of you think? Would a little taste of disaster followed by a heroic rescue improve Bob's relationship with his client?

Also read these IT Consultant posts:


Chip Camden has been programming since 1978, and he's still not done. An independent consultant since 1991, Chip specializes in software development tools, languages, and migration to new technology. Besides writing for TechRepublic's IT Consultant b...

bkindle 1 Like

I was involved in a similar situation once and it's always best to be honest and proactive as much as possible. Some clients will recognize it when it happens and thank you for your efforts to resolve the issue, and some need to be fired as clients for treating your consultancy like a loss instead of an investment. Of course, fire them after receiving that last payment, if your lucky they pay their bills promptly after making comments like the one mentioned in the article.

reisen55 2 Like

Never would have I have let the server itself remain dead for Monday morning, that is not what I do or am paid to do for any client. As it was, I was partly lucky in being able to on-site trash a 500gb SATA drive for a few days and spin it up. But the odd thing is that the client per se did not suffer a server crash - I suffered the crash and repaired it. And the drive was toast, I ran diagnostics on it and it was gone. As I have tested disaster recover here many times, recovery was not a panic issue at all. I am trained in that, a direct participant in the World Trade Center. My servers crashed 103 floors, South Tower, which is a non-recoverable event. I also like to be sleeping at 2am and not rebuilding servers at 2am. I test, document and the periodically update. But never would I actually enact a failure so the client could taste it. But it an oddity of our business is it not. "Just what DID you do to earn this invoice? I see nothing wrong this morning." Oh, isn't that the way it is supposed to BE??? Weird world.

Sterling chip Camden
Sterling chip Camden 1 Like

... retainer-based fee structure. "Pay me to make no news the good news."


Oh yes, I am far down the road here - 14 years in this office in fact. I know this place inside and out. And if I really wanted to cause problems, that would be easy of course - a server crash is NOT the thing to fake. I also did not cover STRESS for even though I had it all back in 3 hours that day, it took a helluva lot out of me and I still had to take a long walk on the way home just to cool off. I do not recommend server crash for the faint hearted.

Charles Bundy
Charles Bundy 1 Like

Only adequate or inadequate preparation for a statistical universe." R.A.H. Bob's colleague sounds like Bob's client. Both comments exhibit insensitivity to Bob's professionalism. And systems admin and technical work are a profession. As IT has become cheap, friendly and pervasive those who take care of the infrastructure get treated more like Chuck the Janitor. Only noticed when the floor isn't clean, and how hard is it to clean a floor anyway? In Bob's case it was his personal attention to detail that allowed a disaster recovery scenario to play out successfully. But the client's comment certainly shows a lack of regard for the disaster recovery planning process. I like Chip's suggestion of going over disaster scenarios, but would add that this should segue into a disaster recovery planned process. I much prefer communication over sabatoge to get my point across :)

Sterling chip Camden
Sterling chip Camden

Bob's colleague essentially advised him to fight the perception of a lie -- with another lie.

skris88 1 Like

Customers think they KNOW information technology. Hell, they can fix their PCs most of the time (cannot open PDF, downloaded Adobe, and 'fixed' it), right? So they will always consider IT personnel a 'wasteful' expense. We just have to deal with it. The number of time I have been "accused" of deliberately infecting a clients PC or network so I create more work for myself is in the hundreds. But that is the life of the industry. I just take it in my stride. We're damned if we do, and damned if we don't....

Alpha_Dog 1 Like

I can see where you may wish do do things on the list and more... it even takes us to our happy spot, but to act in this way is a clear violation of professional ethics.

dthomas1818 1 Like

I'm confident we have all heard those kinds of statements and it can wear you down at times. We had a few (and I do me mean "had") customers that would make statements like this everytime we came through the door. We as a business came to the conclusion that these kinds of customers didn't understand the value of our services or even attempted to undervalue our services. We had an open and honest conversation to this extent and in some case, we agreed to go our own way and in others, the trust and bond between us has grown stronger. Here is something we adopted a few years ago as a consulting company that has paid divideds over and over. It can be a hard path to follow, but it has been bullet proof and worked in the long run...... Know your customers. Is your customer staring at their watch the second you walk through the door? Do they attempt to bargain/barter with you? Do they not sign on to some technical support agreement? These indicator questions tell us what they think of technology. Some customers don't understand the value and for us, they aren't customers for us. On the other hand, customers who do understand our services are quick to get on board. After a short period, we occassionally will get the comment, "since you all have been servicing our IT needs, we haven't really had any issues and we really see the benefit and value of what you firm does for us. That statement means the world to us and it has been stated on several occassions. A couple questions you can ask yourself. If the the problem customer told you they were going to seek other technology firms for service, would you be unhappy about that statement? The other question you can ask yourself is, If they came back six months later and asked that you take them back as a customer, would you take them back? I know it's very difficult to let a customer go, especially from a sales perspective, but what we have found out is that we spend 90% of our administration on 10% of the customers and we have concluded it's those same customers that hastled us to no end. We were loosing money by trying to get them to change their approach to technology. Today, we can identify those customers immediately and we steer clear of them. Our business is booming better than ever and we spend all our time taking care of customers who truly value our services and in turn, they get superior service from all of our technicians all of the time.

Sterling chip Camden
Sterling chip Camden

Agreed wholeheartedly. It's the same principle as in software development when you're trying to include every feature ever requested. The difficult minority is often best left out in the cold.

reisen55 4 Like

I have sometimes told my clients that I am the fellow you pay NOT TO SEE. Curious statement, but say we have two consultants in different scenarios. One is working like mad, solving problems, billing hours, nothing is going right, virus intrusions all over the place, etc. WOW, IS THAT GUY WORKING HARD!!! The client thinks so even as the client is writing monsterous checks. Boy do we need him on site. Second situation - a local area network properly managed should be BORING without too many problems, and running well and protected from failure, as my situations generally are. So here the consultant is indeed doing his job but in the background, monitoring, correcting BEFORE mighty problems hit. It is a managed network. Indeed, in corporate support I am partly convinced this is the perception for outsourcing. Why do we employ all those server administrators? They are doing nothing. Oh, Really!! True you do not need 17 administrators but maybe the few you have really ARE doing work, but not the TOTAL PANIC QUANTITY kind I described in scenario one. ProActive management is generally invisible to the client and staff,as it should be. And yet, look at the subject of this blog. Go figure sometimes. Damned if I do and Damned if I don't.

AnsuGisalas 2 Like

for they know not what they do... Imagine a situation where a Corporate is saying "Yeah, so we got rid of our old server admin, he was no good - never doing a damn thing... but you should see the new one we got, wooo boy - he's working so hard we need to mop his sweat off the floor!" Is that funny or is it sad? Or both?

Sterling chip Camden
Sterling chip Camden 1 Like

Lines of Code produced per day used to be considered a good measurement of productivity. As Dijkstra noted, that metric should be inverted.

JohnMcGrew 1 Like

Yes, I've dealt with this. I can't imagine any consultant who has not. I've got a few clients who I have so successfully sheltered from "the real world" that they often openly question and resent bill for all of the preventative measures I make sure happen on the proper schedules. Their computers book up quickly every morning, their databases are secure, and they're protected in case of any true disaster. The irony is that at the same time, they all personally know others who do not employ any oversight over their IT needs, and regularly have to replace their computers simply because they get malwared-to-death every year or so. There's little you can do about it, other than to remind them the consequences. Fortunately, there are enough high profile screw-ups that they see in the news to remind them that perhaps I do know what I'm doing, and am worth the money.

Sterling chip Camden
Sterling chip Camden

I think that's key. If you've demonstrated your trustworthiness over the years, the client will be more likely to view uncertain cases in your favor.

Calcom Tech
Calcom Tech

If your mission statement and personal conviction matters to you at all, in this situation I believe being truthful should be the first choice in any situation. What could occur is that you add a ignorant charge to the bill for the comment or charge a off hour/weekend rate to somewhat compensate for the nasty remark. That remark by the way was nasty and not called for but we have all had it happen, at least I have and I responded the way I just mentioned.

herlizness 2 Like

When I'm in the customer role (for anything) in a repair/maintenance situation I tend to ask a lot of questions when technicians and servicemen say something needs to be done, particularly expensive things or things that "shouldn't have failed." When I get good answers, I'm ok with the recommendation, go with it and pay for it. When I hear hedging, fumbling or just plain nonsense I'm not ok with it and bring in someone else. The problem with this is that there are some people who are very good at fixing things but absolutely horrible at explaining why it needed fixing and how they intend to fix it. The reverse can be equally true. Moral of the story is to be good at explaining and/or demonstrating the course of action you're proposing or have already taken. It tends to work. But, there are some people who are just paranoid and don't believe anyone; you should probably drop them as clients whenever possible.

Sterling chip Camden
Sterling chip Camden

... some vendors probably find you highly annoying, they'd do well to appreciate that you aren't resenting and suspecting them behind their backs.

AnsuGisalas 1 Like

you have a clue. Hanging around here, you know more of the way of things than most of these people's clients, many of whom might only know that the machine is working... or isn't working, as the case may be.

loren.saunders 1 Like

If you do your best by the client and they make crack jokes like that, then they are not doing their best by you. Find another client to replace them, then let them know that you can no longer work with them and they should find a different consultant to help them out. It always pays to diversify your client load so that you're not dependent on one fat client, so that if this situation occurs you have the flexibility to deal with it. Very good article topic... this happens to all of us consultants I'm sure.

viProCon 2 Like

In the 2nd last paragraph of the article it says to plug in the drive to prove it's dead. I have to disagree with doing that becauase how many times have IT Pros run into situations where something that was definitely dead, came back to life even if only temporarily? Then you find yourself backpedaling, trying to explain to non-technical customers why it is that soemtimes something that is dead is only half-dead and can work partially under different circumstances. Example, the hard drive in question might be failing due to over-temp issues - but if you power it off, let it cool for an hour, then plug it in again maybe it'll work for a bit. The examples are endless but the bottom line is that I would advise that you must teach your clients to trust your diagnostic capabilities - if they don't, find ways to teach them but there must come a point where they can just accept what you say because you've built up that trust. Btw, about that "teach them a lesson" thing. I agree they need to know how close they came to disaster but I don't believe willingly letting disaster befall them is ever a good thing. At least none of my clients are the kind that need this - but I can imagine situations where a client is just too ignorant to try to understand how fortunate they were in a situation like this article states. Nobody these days should run a business that utilzies technology without being obligated to understand the basics of it - or failing that, to simply let you do your job and trust in what you do.

Sterling chip Camden
Sterling chip Camden 1 Like

You're right, that would be the one time in the rest of history when that drive worked flawlessly. Maybe present a console log of the actual failure instead.

tbakken 1 Like

bottom line, your integrity is what matters. trying to live in the limelight all the time means you have to try and look good. if you are good then you don't need to worry about it. it's all about looking at yourself in the mirror knowing what you see.

tbmay 1 Like

I'm with you there. This is a great discussion about the challenges of not only freelancing, but working corporate. People don't value your expertise. They want to equate it with solving mechanical issues, and it's not even close to the same thing. I've rebooted my consultancy. I'm working support agreements only for any "fixing" of things. I do non-supported project work, charging by the day, and that work is second priority to any needs of supported customers. Lost customers in the transition, but the ones that hire me understand the value of a technology professional to their business. Ultimately, it's not in me to "let them have a taste of catastrophe. If they hire me, I take care of them...proactively. If they don't, I'm not the guy to call.

martingolden 2 Like

In all instances we must do what we know to be correct for our customers. I cannot imagine any other way. When your customer under appreciates the excellent IT services provided, certainly educating the customer is in order.

info 2 Like

Chip, I think you're right. Being truthful is the only way to go about this scenario. A discussion after the fact is the best way to let the customer know what could have happened if you weren't there onsite when the problem happened. If they don't appreciate your work for them, then maybe it's time to find another customer who is more appreciative of your work! David

jeffreybradley 3 Like

All the work that was done to prevent disaster was blown off as see nothing happened anyway - basically the same perception.

Sterling chip Camden
Sterling chip Camden

... about how much of it was really needed. On the one hand, you can argue that if we hadn't taken it so seriously things really would have ground to a halt. OTOH, some consultants probably did make work for themselves.

Sterling chip Camden
Sterling chip Camden

... that you were making business for yourself? How did you deal with that?

ChrisEvans 1 Like

Professionally - the only way to ever deal with a customer, and just laughed it off. This article does however bring up the serious point of customer understanding. In the same way as you would do DR testing, it is essential that your customer understands the pain of failure when making decisions on investment in mitigation measures. I am not suggesting anything so serious as kicking the power out of a server but certainly considering simulation or at least bringing home the real financial cost of the failing must be done in order to drive the point home otherwise you just come across as a scaremongerer trying to drive up sales orders.

AnsuGisalas 1 Like

If a consultant was to offer, perhaps even as a free service, to make an evaluation of the costs of critical failures of various kinds... not necessarily in dollars, that might require info the client doesn't want to share, but in estimated down-time. Especially fun would be to show the dependencies involved, as often I gather, a company may be unaware of how one machine failing can impact other functions. The company can easily convert downtime of each business function into dollars, they'll get that point. The trick is that the evaluation can be used by the company to make educated choices on how to prioritize purchases - and simultaneously brings the dollars and cents the consultant can save into view. Sort of a benign trojan, knowledge that shapes the thoughts of the recipients in a certain direction. Of course, due to this, it'd be important to make the report very impartial and devoid of preaching. The preaching works better when strongly disassociated from this, more devious manipulation. If it even is a manipulation, this is something the client should know anyway - it just so happens to serve dual purposes. As Macchiavelli would have said: any act which ensures that the principality does not rid itself of good counsel - is a good act. In this case, since the consultant is good for the client, making sure the client knows to appreciate the consultant is in fact an act in the best interests of the client :D :D :D

Sterling chip Camden
Sterling chip Camden 1 Like

Yes, and I think you have to develop a history of impartiality. They need to remember cases when your recommendation specifically benefited them instead of your pocket, or else no matter how realistic the information, you sound like an insurance salesman.

Editor's Picks