First this: "Ok... I'm ready... now, just press the Enter key and I'll be done."
Followed by this: "Uh, oh." or "Oops." or maybe something a little more colorful.
It's happened to all of us: That sinking feeling in the pit of your stomach when you do something boneheaded and the realization hits. No matter how hard we try or how many safeguards are put into place, we're all human and mistakes will be made. I thought I'd take a break from the mundane world for a bit and share with you two of my doozies. Now, I'd prefer to be joined by some of you in this! Share your own stories in the comment section.
And now, for the show:
- Database disaster. Early on in my IT career, I was one of two people — the other person was my boss — that managed a few dozen database application for a few K-12 school districts. Each of the applications was linked to the primary student database. For flexibility and ease of maintenance, we didn't physically copy the student database to the working directory for each application. Instead, we created a symbolic link in the application's working directory; this symbolic link pointed to the main copy of the student database. During the development of a new database application, I had to establish symbolic links to the main database. Early in the process, I decided to move the new application to a different directory and I had to delete the newly created symbolic links. Well, somehow, I managed to change to the directory that contained the primary copy of the student database before issuing a delete command. Long story short, within seconds of hitting the enter key, the phones went berserk as I had deleted the database out from under about seventy applications. I had the problem corrected within about an hour, but felt pretty small!
- Network nuisance. About ten years ago, I was the network manager for a small college in upstate New York. At the time, the college was using a Fore Systems (now Marconi, I believe) PowerHub. In order to add a network segment to a VLAN, there was a relatively convoluted command that had to be issued at the PowerHub's command line. At one point, I needed to add a port on the PowerHub to the IT VLAN, which housed all of the College's servers. Since the required command wasn't something I did very often, I consulted the PowerHub documentation, carefully followed the command syntax and made my configuration change. Immediately, the workstation I was using stopped communicating on the network. Realizing that something was amiss, I, quite literally, ran to a computer on a different VLAN and looked at the PowerHub configuration. Upon looking at the IT VLAN details, I saw that the port membership of the IT VLAN included only the port I had intended to add to the VLAN. All of the ports that were originally a part of the VLAN were missing. I quickly recreated the IT VLAN to correct the problem. Afterwards, I contacted Fore Systems and asked them about what I had done wrong. Their response: "Oh... our documentation has a typo. It happens all the time."
There are always lessons to be learned from mistakes and these two were no exception. Obviously, I learned to check things more carefully before commiting an action.
Now, it's your turn! Share some of your boneheaded mistakes below.
Since 1994, Scott Lowe has been providing technology solutions to a variety of organizations. After spending 10 years in multiple CIO roles, Scott is now an independent consultant, blogger, author, owner of The 1610 Group, and a Senior IT Executive with CampusWorks, Inc. Scott is available for consulting, writing, and speaking engagements and can be reached at firstname.lastname@example.org.