Last week was a good week. We got a lot done. We stabilized a system that was causing major problems. We identified and addressed some key problems, which will in turn slowly ramp down the incidents we face on a daily basis. We also got our rear-ends kicked by a hack in a server we didn’t know about. It took us two days to bring a critical system back into a stable configuration because someone five years ago came up with a clever work around and didn’t bother to tell anyone.

I have a confession to make. I hate hacks. I hate the little tricks people pull with software to “get it to run properly”. I’ve used them, worked them out myself, even looked the other way when, in a crunch, my teams deployed a system held together by them. But I still hate them with a burning passion.

It’s not that I don’t appreciate a good solution to a puzzle. Honestly, I like the little thrill in my heart when I come up with a sneaky way around something stupid software does. I enjoy seeing the servers go up, providing stable service to my customers. I like finding the answer. Heck, I like seeing other people find the answer and letting them prove just how smart they are.

Those thrills and the status we gain from proving our intelligence do not match the price we pay later on, though. Information technology management isn’t about what happens today. Not really. Other than a handful of situations, today is mostly ancient history. We have to keep our eyes focused on what happens one, three, and five years down the road. What happens to that cool little hack when we apply a patch that breaks it? What happens when we move on, leaving behind an environment running on hundreds of little things no one knows about? What happens when one of our systems breaks and we are laying in a hospital bed, watching the pretty pink elephants and hitting the morphine button?

Every hack, every undocumented tweak to make the system keep running, becomes a liability later on. The right way to do things often takes a lot longer than the short cut. It’s dull and plodding compared to the light-footed approach of “just get it working”. Frankly, I find it constraining at times myself, especially when I’m in a huge crunch to get something done.

The challenge as a manager comes in when I try to work out when, where, and how much to interfere. There exists a fine line between trying to get things done the right way and obstructing the team’s progress in dealing with known or unknown issues. When people get into a groove they really don’t want to stop. When the fire’s coming down, goals must be met, and the “world will end if we don’t get it done today”, then pesky things like documentation and planning go out the window. We make it work. That’s what we do.

I also know once I implement something the odds turn against my bothering to document it. It’s not, despite what some people say, a mental defect. Like most of the people I’ve met in infrastructure work, I like solving problems. I move on once I solve a problem. Heck, in two weeks I might not even remember how I solved it. Goodness knows, there’s enough new problems to keep me busy; even if I remember I will not take the time to document.

So..when to push and when to not? For me it depends the individual involved and the work. I don’t push for documentation about the creative problem solving process itself, though I like documentation about the end results. Any time we get involved with communicating with outside groups, though, I work with the technician to get good, effective communication out into the field. When we get involved with various director and executive levels I try to get my managers and executives involved; it’s their job, after all.

I’ll let you know on Friday how this week turned out. It’s gotten interesting, what with most of the team out sick.