General discussion

Locked

What/how would you document to guard against 1-person failure point in IT?

By Understaffed ·
I am a single-person I.T. Department for a non-profit child welfare agency, managing 150 users, 100 desktops, 6 servers, and all aspects of technology. We have our email, database, app, file, and print servers, phone system (within a couple of weeks moving to VoIP), accounting and case management systems all in-house. The only thing I don't host or manage is the website, due to lack of time and flukey local utilities.

I have not been what I would call "diligent" when it comes to keeping change-logs when servers are set up and apps installed- I tend to keep the most relevant stuff in my head. This scenario, however, is a disaster waiting to happen. If something were to happen to me, 1) nobody in this organization would be able to fill my shoes even temporarily (social workers...) and 2) I would think a new person would have a hard time getting acclimated to the network, due to the lack of *settings* documentation. I did create a "LAN Book" that contains things like server names and hardware configs, LAN physical layout (which is quite convoluted), services and apps running on particular servers, and DHCP/DNS settings.

My problem is this: I built this network from nothing, and this has been my only work experience in the IT field, so I have no others to look to. I have been tasked with creating a manual that would allow someone to run the network in case I die in a car wreck on the way home.

What would you include, and where in the world would you start? To me, this seems like an insurmountable volume of data... I know that my CFO wants a domain admin account password kept in a safe deposit box, but where is the line drawn- do I include that the fiber between buildings is 6-strand 6.25 micron direct plant with SC ends, just so that person knows?

What say those in the trenches?
~Thanks,
~Dave

This conversation is currently closed to new comments.

Thread display: Collapse - | Expand +

All Comments

Collapse -

Not always the case....

by Greg O In reply to DO document yourself out ...

You make an assumption that is usually not true in a "one-man IT shop" in a small business, like I am. In most of these scenarios, and mine, the IT person may work straight for the CEO/Pres/whatever, and there IS no promotion potential. I work in a civil engineering firm, and unless I'm a civil engineer, there's no vertical opportunity at all. I could document almost all aspects of my IT job and the CEO could easily outsource it to a local on-demand service that could do it quite a bit cheaper than my salary, provided I gave them a good outline of my unique systems. I enjoy my job, so its not in my best interest to make it easily replaceable with no other avenue to stay in the company. My boss knows if I'm doing my job when all the users are happy, system is always up, and we're making good money using the technology at our disposal. I have good documented and tested disaster recovery plans, and a few sheets of key info if I'm out sick, but that's about as far as I take it. The only thing I'm "hiding" is the knowledge that my boss could save more money by contracting out my job. If it meant you leaving your job/company, would you tell your boss how to eliminate your position?

Collapse -

Agree Completely

by robertb In reply to don't document yourself o ...

As opposed to some of the other replies on this subject, I agree completely with the idea that you be careful not to document yourself out of a job. Having worked as an IT consultant, I saw multiple instances of exactly this scenario in smaller IT shops. The Boss says "Our network runs smoothly, and everything seems perfectly documented, so what do we need an IT guy for?" at which point the IT person is downsized. I usually got called in a couple months later as things started falling apart (due to lack of regular maintenance, etc. which the old IT person was doing faithfully, but, even when documented, everyone else was just "too busy" to do after the IT person was let go). It was at this point that I made sure the boss understood that the network ran well and smoothly BECAUSE they had an IT person on staff. If you document the nitty-gritty detail of everything you do, the temptation exists (in smaller IT shops) to try running without you.

I would suggest documenting Logical LAN layout, Server names and services on those servers, Admin and service account passwords, all hardware (Servers, Network equipment, Printers, etc.) and Software. Then I suggest drawing up a list of necessary skills required for running and maintaining the network so that if you are ever taken out of the picture (tragic accident, illness, or other reason) the company management knows exactly what type of person they need to hire to take over.

Lastly, and do not underestimate the importance of this, make sure that the managment understands that with the documentation you created and if they hire a person with the qualifications you outlined they will definately be able to take over where you left off.

Collapse -

Not terribly concerned about that...

by Understaffed In reply to don't document yourself o ...

The complexity of the environment, and the level of conciousness of the users would require somebody to be onsite, so outsourcing wouldn't be feasible. And to bring in somebody else, the pay is so low, and we are so far away from any real city centers, that any perspective employee would be certifiably insane to take what they would offer... (think low $30,000's)

Collapse -

take every opportunity to document your self out of a job.

by IT@SFG In reply to don't document yourself o ...

Take every opportunity to document your self out of a job.

You don't know what is around the couner, Screw the bus, what about a promotomion or what if you want to leave.

What if your company is taken over by bigger company with more IT staff. If your job is documented you can easiy move into the the new IT department and take on more responablitliy and do different things. If you are seen as been required to remain where you are you limit your growth protenital.

If you want to leave will you be happy to walk away with the currnet state of play. If you do wish to leave you don't want to spend the last few weeks doing the documentation.

The more replaceable you make yourself the more valuable you become to the company in other ways.

As for the "document everything posts", we all know how to get into the AD , we all know how to add a new user. What you need to document your username / password poloicies and anything unquie to your company that goes beyond the basics. IE our faxing soultion required the last 4 digits of the persons fax number in the fax # field.

Shaun


Shaun

Collapse -

At the first make a drawing off the network

by manager_it In reply to What/how would you docume ...

Dave,

Make at first two drawins off the network, first the standaard overview (servers, router's,) then make a second drawing off the building whith de logical placing of equipment.

then if you have time, make a drawing off your server room. (wireing over view, equipment over view, Take you time whith it (Rome is not build in one day)

Then make assamble a list of main equipment, you can read it out off your drawings.
Gife every item a number between 1-5 (1 not imported standaard equipment, 5 very very imported cost a lots off work to reconfig it again)

Then starting whith all 5's to document this item. (hardware, configuration, reseller, warrenty, adhoc telefone numbers to get support ect) do this for your all 5's

Then starting whith your 4's ect ect.

I think you must document it because if you walking under a train then nobody now's nothing. If you have documenting it the have something to work with.

I had also read to take a student for a practice, a very good idee because he can help you, and can also do a lot off work for you. But he can also read your writeing and then ask if he can understand it.

You also write you are starting whith VOIP, take the time to document at first all the changes you are going to make to make this VOIP project a succes. Then you have your first document all ready, and keep it up to date.

Put your documentations in a map and place it on your desk, if you want to make change's take also a pencil en make the change in your prited copy. Put a yellow paper on that page and make every week are 14 days the time to make the changes in de orginal digital version.

I come at a lot off places (tempory IT manager) where no document's are and al ways starting on this way. After a few months you have your documentations under controll and up to date. And every time you have some spare time you take a next level off documentation

If you have done this, than you can also easy build a recovery/disaster plan for you company. I think its a must for a company like size you say.

Regards

John

Collapse -

Best advice for anyone starting fresh

by ryoung In reply to At the first make a drawi ...

I have read alot of comments on articles and this is right up there with the best advice I have seen. I am going to start this process now. This is why techrepublic earns it's keep with me day in and day out.

Collapse -

Contingency Planning 101

by HA Wizard In reply to What/how would you docume ...

Don't include what someone else already knows - just include who that person (or company) is and how and when to contact them.
The idea to planning for single points of failure is to eliminate the single point of failure - in this case, yourself.
If you contract your dedicated lines from ??ISP, include the contact information for ??ISP (and contract # or whatever else they need to know to escalate). Additionally, if your servers include 3-yr. service agreements, include that in the documentation and the contact information (plus contract #, etc.) to obtain the service.
Your run-book should only include instructions for what you do exactly and only the necessary information for those tasks (Server Specs., Apps., Network Topology, Domain Names, etc., etc., etc.) grouped by criticality to the business.
You want someone to know where to start and what is most important to run the business on a day to day basis - spending too much time on non-critical systems wastes precious up-time on more critical systems.
Too much unnecessary information will make the run-book overwhelming and non-cohesive.
You should do a proper Business Continuity plan and include a Risk Analysis (this will identify your single points of failure) and a Business Impact Analysis (this will identify your critical services).

Collapse -

My Process

by Todder In reply to What/how would you docume ...

While most people would have the fear of getting down-sized when asked to document their knowledge, in IT, it is a necessity. Providing proper documentation is a skill that will benefit you throughout your career. With Sarbanes Oxley and other auditing requirements nowdays, learning how to do this will only help you. With that said, I would create a very secure folder where whomever has access depends on your environment. In your case I would assume yourself, CFO and President. In that folder I would put the following:

- Complete vendor/contact list
- Admin/Service Account Password list
- Network Diagrams
- Application manuals and any special setup configurations for them
- Data Backup scope/schedule/process
- Disater Recovery steps
- Any policies you created (i.e. network usage, security etc..)
- Run BGINFO on servers and have the output sent to this folder
- Phone system details
- Building alarm codes/setup

That is very good you were able to build this network from the ground up. After completely documenting it, you will visually be able to portray what you have accomplished to your current employer as well as show prospective employers down the road what you did (sans the private information in the documentation.)

I agree with the posts that don't recommend 'how-tos' unless there is some very specific in your environment I hope this helps.

Collapse -

I agree; except to add

by tkoenings In reply to My Process

I agree with Todder. The only change I would make is to document from a disaster recovery prespective. What would an IT person (you or someone else) need to know if x, y, z or all the above were to fail? What would it take to get it back to where it is today? If this needs to be step-by-step then write it as such. Otherwise, just include notations. Remember, to update the documentation as you make server/network changes. Finally, remember someone else will problably be doing the wiring and similar tasks, under a complete recovery so only document the basic information. That's my 2 cents.

Collapse -

understaffed and overworked

by sparker In reply to What/how would you docume ...

It seems like your boss wants everything for nothing! I agree with others that you do not need to document simple admin type procedures (adding a user, resetting a password, etc) unless you do them in a strange way.
I would, though, make lists of vendors, customer #'s, passwords, admin passwords, etc...
You should have a wiring map of your network with cable types, etc noted.
I have had to come into situations as a contractor where the "one man band" decided to quit without notice and it was a mess. Documentation is a hard thing to keep up with, but spend a few days walking around with a note pad (or PDA) and write down the things you do. Ask yourself "would someone else know how to do this?"
Lastly, on your next review demand a raise AND an assistant, you obviously deserve BOTH!!!

Related Discussions

Related Forums