Outsourcing

Never assume engineers have mastered all the fundamentals

Erik Eckel argues that it's unreasonable to assume every technician can master the vast technologies consultancies are commonly tasked to support. Read his suggestions for building specific areas of expertise on your staff.

Consulting offices occasionally resemble a triage ward in which numerous clients are experiencing multiple and varied crises simultaneously. The temptation during such firestorms is to dispatch any available or the closest engineers to troubleshoot and repair the issues. But because failures are so often due to such a wide range of issues -- a failed DSL modem, a toasted router, a corrupted Exchange store, a virus-laden workstation, a failed RAID array, a bad battery backup, etc. -- you can't.

Consultancies can't assume every engineer has mastered every fundamental. For example, it's probably not a good idea to send the SonicWALL expert to troubleshoot a corrupted SQL database. Consultancies will get calls about Microsoft OSs, Linux OSs, Macs, software applications, routers NOSs, managed switch configurations, physical plant errors, backup failures, cloud-based app failures, tainted databases, failed telcom pipes -- that's two much for one engineer to master. I learned the hard way.

The problem may be prevalent at consultancies that leverage corporate IT departments as staff feeder systems. Many consultancies recruit engineers from the corporate ranks. Within corporate environments, engineers frequently become extremely proficient performing a select group of tasks. Chances are, those engineers didn't have to support or troubleshoot the tremendous variety of equipment, OSs, applications, and routing and network systems a consultant does when working for a consulting firm.

Stop don't assume (or hope) all your consulting office's engineers have mastered all IT fundamentals -- that's unrealistic. Instead, consider building specific areas of expertise on your staff. Encourage technicians interested in routing, VPNs, QoS, and the like to gain SonicWALL and Cisco certifications. Help database administrators find a rack server you can repurpose to let them study the newest SQL platform in a nonproduction environment. Assist another in mastering all aspects of Microsoft's new Exchange platform.

Sure, everyone in the office needs to be comfortable performing basic hardware and Windows troubleshooting tasks. Everyone needs to be able to remove viruses and spyware, configure new user accounts, and repair failed smartphone and Outlook configurations. But not everyone needs to chase QuickBooks point-of-sale certification, earn Cisco accreditation, or prove masterful troubleshooting Dentrix, ACT!, and MAS 90.

If the only engineer available in a crisis isn't certified in the technology in question, the tech can still go to the client site and eliminate obvious errors, isolate the issue, and serve as the resident expert's boots-on-the-ground, if needed. As long as you have the knowledge you need on staff, he or she's only a telephone call away. Resist the temptation to criticize the on-site tech, because it's unreasonable to assume every technician can master the vast technologies consultancies are commonly tasked to support; that's the consultancies' overall responsibility, not the single technician's role.

About

Erik Eckel owns and operates two technology companies. As a managing partner with Louisville Geek, he works daily as an IT consultant to assist small businesses in overcoming technology challenges and maximizing IT investments. He is also president o...

23 comments
mwclarke1
mwclarke1

My biggest problem with trying to effectively troubleshoot issues is when you get a few VP's, other executive management crammed in your office/data center trying to get a status every 5 seconds of what the problem is, when will it be fixed and in some cases trying to insist they know more than you and try to derail your efforts to the wrong side of the issue! No joke, I had 30 people crammed into a data center hovering around me and another staff member as we just started to look at a major outage issue. It quickly became chaotic, The data center manager asked if I needed anything, I paused for a second, turned to him, and quietly told him that if anyone in that room knew how to fix this they need to be siting here or get the hell out. That room was cleared so quick.....

Alpha_Dog
Alpha_Dog

The truth is that where the prey is too big to handle alone we had to adapt and become pack hunters. Today a job gets too big we can call in the troops or get flattened by the mammoth.

Tony Hopkinson
Tony Hopkinson

Would you send an oracle guy to deal with a corrupted SQL server install, or possibly worse still vice versa. In some ways they might be worse choice than your cisco guy, at least he definitely knows he doesn't know anything about it... You could send either of your dba's to figure out why something was slow though, they might end up calling out the network chap, if it was client to server... The fundamental any generalist or specialist in anything needs to have troubleshooting, and that's so far from a given it's not true. The thing is if a client has problem where there are a number of possible areas where something could be wrong, sending an expert in one of them is troubleshooting failure number 1, you just guessed where the issue was and worse still by sending your expert in X, he's going to assume it is X that's at fault. If they aren't a good fault finder, they'll be shaking the machine to dislodge the gremlins instead of making sure the printer is turned on.... The assumption you are talking about is one prevalent right across IT amongst the non-technical, it's why people come up to me as programmer and ask me to sort out an active directory problem, or a PBX, or fix their scanner... The above is the first thing a generalist should know, and it's a good generalist you should send first so then they can call in the correct specialist. So as usual it would seem you've got it bass ackwards again, you should be making sure your specialists have the fundamentals, or employ some fundamentalists. :D

Alpha_Dog
Alpha_Dog

We need to make sure we don't assume that we have mastered it all either. There are times our craft must seem like a black art to the average user. Our inexhaustible depths of knowledge of all things arcane never fails to impress. The trouble is that sometimes we start to believe our own press. The correct answer to many situations is "I don't know, but let me find out". If our pride keeps us from uttering those words, instead muttering something about sacrificing a virgin to the SCSI chain, we need a reality check or a new career. One thing is certain, if the customer discovers the man behind the curtain, you are done, no matter how much amplification or smoke you use.

Alpha_Dog
Alpha_Dog

We have developed a set of rules relating to this, and much of it comes from my dad who ran a communications shop after a time in the military. First, in an emergency never pass up an opportunity to have boots on the ground and eyes in the field. It may be that the SQL/Oracle server isn't on for some reason. Nearly anyone can troubleshoot this if they are there. Second, make no changes which have to do with substance unless qualified. Checking a plug is one thing, but reindexing? Leave that to the pro. Third, if it is not an emergency, take the time you need to marshal the resources before attempting anything. Is it the backup test server? Send the proper guy with the proper training, not the guy who is closest. Finally, Everyone should have the same foundational knowledge. This is what the A+ is for. Common knowledge not only makes everyone a stronger tech, it also makes communication easier and more efficient. There are other rules, 12 in all, but those 4 are the ones germane to the topic.

santeewelding
santeewelding

When I scale up what you say beyond IT, and apply it to your comment, I wonder where it is that you, yourself, are coming from. Yes. I read Erik's piece yesterday. His was from the trenches. Yours is from...a mount? Good point you make, though.

Tony Hopkinson
Tony Hopkinson

is my rule one. Not until I understand the problem anyway, and that comes from proving the scenario without making a change you can't back out of without a side effect. Always check your assumptions before you act on them is rule 2. You ever worked with one of those people who's fault finding goes like I think it's this, change it, sonething else goes wrong, oh right I need to change that as well now , oh and this, and yep a bit of them.... FFS STOP! You first change was wrong, oh what was it I did again? In the systems we work in every change you make has an effect, it's meant to be fix the problem, not introduce three more. I got taught the real art of fault finding by electricians in heavy industry. Strangely in their approach at no point does it say stick your screwdriver in there and see what happens. :p

AnsuGisalas
AnsuGisalas

A hint : We, the consultancy versus I, the consultancy. Explains a lot of the chasseeing in the mine field.

Alpha_Dog
Alpha_Dog

I have occasionally been guilty of the whole 'believing my own press' thing, but not recently. This is one of the many things I have my competition to thank for. In the process of taking on things he's not good at, he gives me life-long customers when I step into the job and fix his screw ups as well as giving me a vivid example of what not to do. He would be better off saying "I don't know, but I know someone who might" and refer the client to us or someone else for this issue instead of turning up the flim-flam until the client calls BS and sends him packing. While I am grateful for the business he inadvertently sends my way, he frustrates me. Wasted potential is a pet peeve of mine... I can't stand it, even if it benefits me. He is better than I am in large scale and high availability Windows networks, while I am better at UNIX and general IPv4 network infrastructure. He tries to do it all and fails miserably when he could partner with two other companies in the area. We could all benefit from each other's experience and improve all of our clients' service. As far as my post being from a mount... I am in Colorado and over a mile up, but I will admit the soap box was perhaps a bit over the top :)

Alpha_Dog
Alpha_Dog

I have an aquaintence who does pro audio repair. He has about to buy a $400.00 tester to determine if electrolytic capacitors were bad in a board without removing it. I taught him how to take a cap approximately half of the value of the cap under test in the board and put it in parallel with the original cap. It the circuit goes wonky (tHD ok, but EQ goes to crap) the original cap is fine. If the audio clears up, humm goes away or the whistle clears up, then the cap under test was bad. The shocking thing is that he couldn't tell me why it worked. I literally had to go into cap 101 for him to sort of get it. Fortunately, I think the technique has saved many power supplies and amps from the trash.

Alpha_Dog
Alpha_Dog

This is true, but when the book doesn't cover the fix, the box goes in the trash. I have rescued and "debricked" more than my share of equipment. I have a fairly extensive computer lab now thanks to the generosity of the "box swappers", and my kids have never known live without a computer.

Alpha_Dog
Alpha_Dog

But they are a whole lot less fun for the audience.

Tony Hopkinson
Tony Hopkinson

weight they put behind sensible fault finding, even with the accessible discrete component going down, they aren't big on guessing..

sboverie
sboverie

I agree that understanding electronics is an advantage; I find it weird when a software support tech blames the hardware for certain problems; obviously he does not understand how the hardware works. A good electronics background does help when you need to fix a problem with limited resources like manuals and test equipment. But, for the most part, the repair has been dumbed down so that techs replace black boxes instead of knowing what the black box does.

Tony Hopkinson
Tony Hopkinson

Is build in points where you can safely attach a lamp. This stuff isn't rocket science, not taught in CS though, nor is fault finding / debugging. As though it was obvious, but as we all know common sense isn't that common....

delf20k
delf20k

Test lamps work better that screwdrivers .

Alpha_Dog
Alpha_Dog

Very few techs these days have the electronic troubleshooting skills us old farts have. The truth is that we don't really need them directly, but the understanding of the principles helps to make informed decisions in the absence of a service manual. You sure there's no approach that involved jamming a screwdriver in there? I was certain that I saw approach demonstrated by a technician on the back of an old Spelman lab power supply directly over the label that said caution high voltage. The only reason that was funny rather than tragic was that the current was minimal... man, was it funny though. He couldn't talk right for three days! It took him a week to get fired because he couldn't hold a pen to sign the forms until then.

AnsuGisalas
AnsuGisalas

It's so easy to loose sight of the forest for trees... This thing we call culture, civilization, fellowship of man, it's the only reason we're around today. Cooperation - even helping the other one up the cliffside, so he or she can drop us a rope. Our true evolutionary niche is running after prey, tracking it till it drops. And we revert to that, at times, or sometimes we get stuck in that mode. Then, when we give in to the stress and start seeing all other people as competitors, and in our way... that's when we shrivel up. The trick is, like your story tells, is to team up with other people who understand this, who know how to take and to give back, and not with the runners-after-antelope, who only can take, and then take more, like locusts, until the ground is dead. Sadly, many many capitalists are of the latter kind, the world would look entirely differently if they were the former kind, and they'd be richer too.

Alpha_Dog
Alpha_Dog

This has been the biggest issue for us internally and for myself personally. The reasons for the identity crisis will become abundantly clear. The story is a bit long but with your background, which I sense also has some psychology in it, will explain much. I became a we in 1993 and back to an I in 1997 when I moved out of the area and my team kept everything going under their flag. In 2000, after I had enough of the mindless reactionary corporate culture around Y2K, I became a we again, though it was to be short lived due to the economic downturn following September 11th (a strange time). I became a we again when I merged my company with another around the spring of 2002. By '04 I had enough of the partners with a terminal case of executivitis, sold my interests and moved outside of the non-compete radius of my contract. In '05 I became a we again when I partnered with another one person shop to bring high speed internet to rural areas (WISP). I resigned when I realized that I was doing all the work, not getting paid, and being accused of the embezzlement she was guilty of. I continued on as an I and in '08 became a we again with employees. By '10 I realized my office manager was a better CEO than I could ever hope to be (imagine a 50 year old tech savvy Pepper Potts). After her divorce I happily demoted myself to CTO and she became CEO. Today WE can accomplish more because the CEO can do the office, paperwork, and legal crap, while the CTO can actually (finally!) do what he's good at. Best decision I... err... we... ever made. To be honest, realizing that the woman I hired to man the phones and file paperwork was CEO material whereas I was not was a humbling experience. I am very glad my idiot competition was around as a poor example, otherwise I might have studiously avoided this fact and tried to do it all myself. I may take him to lunch for that one of these days, just to balance the books for myself. The fact that it'll confuse the hell out of him is just icing on the cake :) As I said in a previous post, untapped potential is a pet peeve of mine. It literally makes me twitch. The hardest part was getting the woman who had all this potential to realize it herself. She had constantly been belittled by her abusive husband, another IT person, for years and she had little self confidence. The day when she realized she could do something her husband couldn't was the day she hit the gas and left us all behind. Back to the point. I think many of us try to be everything because we alternately feel that we are the only competent person in our hemisphere or that others may be better than we are and could steal our income. The truth is that we can't know it all. There is too much. We have to specialize and in so doing build relationships with others who specialize in those areas which we, out of necessity, neglect. By combining into a collective, both in the organization and outside it we become greater than the sum of individuals which comprise it. As Aesop said in his fable about the bundle of sticks, "In union there is strength".

AnsuGisalas
AnsuGisalas

it's a mindjob alright. By way of an introduction, I'm a linguist by training, and I did a lot of work (for naught) on collective and distributive concepts in various languages. A consultancy may in fact collectively hold vastly diverse competencies, but it doesn't follow that each member of the consultancy gestalt distributively holds all those competencies. Which, I guess, is Eriks point too.

Alpha_Dog
Alpha_Dog

Yeah... The I recently grew into a we. It's taking a bit of getting used to, especially the part about not being the boss of me anymore.

santeewelding
santeewelding

I could have used that word, years ago, in relation to a minefield, working out how all is. "Swamp" was the provisional I settled on. What were we talking about?

AnsuGisalas
AnsuGisalas

"How to share lessons learned the hard way, and not sound preachy" Erik is learning it the hard way ;) This piece of his, on the level, is equivalent to the infamous pros/kiddie league comparison, but he's managing to serve it up in a semi-palatable way.