Discussion Locked

Computer Operations 24x7 Coverage

0 helpful
9 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

I'm curious about how other Operations Managers man their computer rooms to accommodate 24x7 coverage. My computer room is a mix of one production mainframe, seven mid-range UNIX servers and 56 NT servers. Operator dities include running jobs off of a schedule, backups, printing, and support. We are currently covered 24x5 with on-call assignments for the weekends. The work days are divided into three shifts with the day shift covered by two operators and the back shifts covered by a single operator. My company has run Computer Operation in this maner since the first computer was installed 20+ years ago.
With only four operators coverage for vacations, sickness and training poses many scheduling difficulties. Going to a 24x7 using this schema seems to me be problematic. I would like to see possibly two operators per shift and move to a 24x7 coverage condition that is not cost prohibitive yet workable.
I am open to any suggestions.

Dan

9 replies

by e-fellow

Forget the past and make sure that you have all the information that you need to make a decision.

#1. Do you have a log or helpdesk that can show you what really goes on in this computer room.

#2. The computer room is a business and it shouldbe treated as one, so map out what you need and then do some simple time studies to see what you really need.

#3. Research software and hardware to automate these jobs, backups, and printing and implement a few and try them out. If some work thenre-map and re-analyze.

#4. When you think you have got it as lean as it can be, think again.

When I started at my current company we had several major IT projects, one of which was a huge ERP/MRP system implementation that had been dormant since its purchase 2 years prior! System personnel were doing everything for everyone and support calls were ridiculous everyday. After 19 years of BS. We have transformed it into a leaner and getting even leaner department. The huge projects of the past are history, now we have new ones. The Helpdesk we wrote is helping us to concentrate on the real issues and still meet project deadlines. We run two manufacturing shifts Internationally and we now have no night shift or weekend MIS personnel and of us 3 in the U.S. we have been paged about once each in the last 6-months. Our Mexico facility has 2 people who work days only and are on call, but are rarely called. However, we still believe we can do better! ?If you always do what you?ve always done, then you will always get what you?ve always got!?

by zakfire6

Dan,

I gree with the other response indicating you have to review your process and determine what support is really necessary. Once you determine the needs, you can then decide to manage this yourself or potentially look to the outside for additional support. The company I represent has nationwide support available. With our size, we can have people available onsite 24x7 and can have personnel to fill in for the vacation, sick time, peaks, etc.

Write back if you would like to discuss further.

Mark

by detailedanalysis

Mark,
I understand what both of you are saying. The problem I am facing is that being a shop that is mixed (NT servers, Novell, Unix and UNISYS Mainframe) poses some scheduling difficulties.
1) The mainframe requires jobs be entered and monitoredon each shift. Mainframes tend to be more operator intense primarily because of the old legacy applications that run on them.
2) Unix, NT and Novell utilize their own scheduling methods and require little manual intervention.
3) The Help Desk activities in my company are handled by a different group from 8-5. 4) Out of normal hours Operations receives the Help Desk Calls. In Cases where the Operator cannot solve the problem, the Help Desk "On-Call" person is contacted. (I personally would like the Operations and Help Desk functions combined).
I was hoping that by posting my original question, I would get responses that indicated "How" other businesses with a similar mix of OS's are organized. Kind of a like a Cyber Visit to their data centers.

Questions I have are:
How best to cover 24x7 scheduling e.g. 3 8's or 2 12's or some other hourly schedule.

Number of personnel needed to cover the schedules to provide adequate overlap/coverage.

Maybe I'm asking the wrong questions.

Dan

by e-fellow

CROSSTRAIN AND BUILD A SMALL M.I.S. KNOWLEDGEBASE OF IS ACTIVITIES TO SUPPORT EACH OF YOU. AND AGAIN AUTOMATE, AUTOMATE, AUTOMATE AND REMEMBER ANYTHING THAT CAN NOT BE AUTOMATED, ACTUALLY CAN BE. GOOD LUCK GUY.

by ken.brown

First off--I can only directly talk about my workgroup. While we are a "operations" group, we don't do: the tape swapping (outsourced/library), run old printers/print jobs, etc.

We have 9 tech's + 1 manager. We have two people on call 24x7 at atime (primary and secondary).

My group (I'm not the manager) directly supports 130+ servers (mostly NT, but some Unix) that are cross-corporate in nature (60,000+ employees). I.e. DNS, WINS, DHCP, Email, firewalls, PDC's, proxies, dial-in, fax, etc. We also provide 2nd level support for other issues/software that is "corporate" to the rest of the corporation. (Email clients, OS issues, modem issues, etc.)

Our backups are automated (although we verify they are working) to a tape library.

We have a more-or-less centralized help desk/call routing group...we don't take calls directly and we only work with technical folks (not end users...except in very few cases...like CIO issues, etc.)

Our "core" hours (8x5) and must have on-call coverage on-site. After hours, the primary person deals with new issues, although both primary and secondary get the pages (alerts and/or help desk cases).

Personally, I'm working on trying to get this changed. I'd rather have the primary/secondary cover 7am-7pm (about) and have two other people cover after hours (7pm-7am). If an issue comes in after hours, the 3rd and 4th deal with it then are not necessarily required to come in the next day.

With our current method, both primary and secondary get pages 24x7 (thankfully not too many early morning pages) but are still required to be in bright eyed and busy tailed, so to speak...which gets to be unreasonable.

Our mainframe support is outsourced. We have a different groups that support different platforms (Unix, AS/400, etc.). Also, a group that only supports the network (routers & switches).

by trina.spalding

I manage our Computer Operations department for a Healthcare Organization. We are 24X7; staffed with 11 Computer Operators. I implemented a 4X10 work schedule in July, 2000. This has worked well because I have enough staff to split up. My 1st shift is 3 people, 2nd and 3rd shift are 4 people each. You can have 4X10 with 2 and 2 teamed together on 2nd & 3rd. It works also with 3 people becuase 1st shift does not need more than 1 Operator on some days. Also, the shift times overlap by 2.5 hours so that staff can communicate to each other, etc. I ask Operators to talk to all shift members before submitting vacation to me so that coverage is already established. I also ask that staff call-in within 3-4 hours before shift time if sick;in an effort to ask another Operator to stay over or try to get coverage.
We too are in a very diverse OS environment consisting of Mainframe, UNIX, Netware, NT, RS6000, Dec Alpha to name a few!!! We do backups, restores, printing/report distribution, monitoring too.... I understand where you are coming from Dan. I do agree with others who have said you may have to justify appealing for more staff by sorting out current/future job functions within your Operations area. The helpdesk we haveis on-site 7a-6p; then call center after hours. Operations often becomes the first level "go-to" for problems after hours. The call center does not have any connection to our Network but does use the triage from the on-site helpdesk before callingOperations. Hope some of this helps!

by sfenner

We have a shop with about 30 UNIX (mostly HP9000) and about 100 NT/2K with some Novell servers mixed in.
6 operators + 1 lead + 1 mgr.
operators work 2ea. 3x8 M-F and 1ea. 2x12 Sa-Su. They rotate working every 3rd weekend with the following monday off. This creates a lot of scheduled OT so we're trying to re-adjust. May change to have just 1ea. on some weekdays. The lead works 3rd shift M-F only.
Responsibilities:
operators mostly monitor automated job sched, handle/escalateproblem tickets, answers calls from IT staff.
the lead handles/escalates issues with nightly job sched (where 90% of jobs reside), verifies backups ran, verifies reports distributed, monitors operators' performance.
Our Helpdesk is seperate in IT M-F 7a-7p. After hours is handled by our Customer HelpDesk who routes P1 calls to operators for escalation.
We have a Network Mgmt. Center for customers which also monitors our Data Center. They create tickets for any alerts and send them to the operators for follow-up/escalation.
We are about to take on the responsibility for Production/Change Control. Any advice?

by trina.spalding

I am the Manager of Technology Operations which also includes Security Administration and Production Control. My advice about taking on Production/Change Control would be to make sure you have Policies and Procedures in place from the beginning. Make sure all other IS areas submitting changes to your group understand the "rules" before submitting. Make sure programmers, DBA's have tested appropriately; have some sort of sign-off document(s) for changes as well as after the change is in place. It is also better if all change control is centralized. It is tougher to get a hold on system & application changes when they are done in different groups.

by oscfdez17

Bare minimun you would need a least 8 Operator. Have you considered 3 day work weeks. This will provide you with some coverage during vactions and sick days. I run a Larger data Center I have 16 operator. 8 work 5 day work week the rest work 3 day work. I also have a mix environment

This discussion is closed to new replies.