Change management is a necessarily complex and convoluted process which, if developed and executed properly via trusted measures, can be a real boon to any organization and its continued evolution and growth. Working in technology involves a certain level of irony in that change is inevitable, yet if poorly implemented can pose a significant risk to business operations.
SEE: Power checklist: Troubleshooting hard drive failures (TechRepublic Premium)
Over the course of my career I’ve found change execution one of the most stressful elements of the job. So, I came up with these 10 golden rules to help safely streamline the process for best results.
1. Establish maintenance windows for hosts/services/users
There may never be a good time to reboot a particular host but some times of the day could be better than others. This is never a popular concept with IT professionals, but a server taken down for a change at 4 a.m. is probably less impactful than one taken down at 4 p.m. Identify maintenance windows for systems, services to help identify the safest time to perform any work on them, whether this work may or may not cause an outage. Do the same for users to identify timeframes when they are least likely to require access.
2. Establish a list of standard terms and tasks related to change rollouts to be used in a form outlining the change
The form should incorporate these terms and tasks as well as the details related to rules 3 through 8.
- Change description; what you intend to do. This is the “what.”
- Change justification and priority; the reasoning behind the change and how critical it is to the business. This is the “why.”
- Impacted hosts/services/users; what elements or people will be involved. This is the “who” and the “where”
- Maintenance window during which the change is to be executed. This is the “when.”
- Implementation plan, technical validation plan and business validation plan. This is the “how.” The implementation plan should lay the change out step by step.
- The technical validation plan confirms the change was successful from a technology perspective (e.g. an operating system was upgraded and is running or a network card was replaced).
- The business validation plan confirms the change was successful from a business operational perspective (e.g. clients can connect to the upgraded server and process transactions).
- Also include a back-out plan to describe how to reverse the change and change outcome terms such as “Implemented successfully,” “Implemented but with issues,” “Implemented but rolled back” and “Not implemented due to issues.”
3. Build a fluid mindset for change preparation and planning
Since every change is different, from the small to the major impacts, it helps to ask a series of questions to best identify the risk factors for a change and how to minimize potential damage.
- Will there be an outage, and if so, for how long and who will be impacted?
- Who needs to be informed or involved with this change? Who should approve it?
- Has the change been researched, prepared and tested appropriately?
- How best can you perform a comprehensive post change validation?
- How can you construct a change in the best way possible to eliminate failure?
4. Assess the change for worst-case impact
Conduct a risk assessment to identify what is the worst-case scenario if the change causes impact, and how you can recover as quickly and efficiently as possible. Include this in the change form.
5. Streamline the ability to perform an immediate back-out where possible
If problems were encountered, identify how best to back the change out immediately and with the least amount of impact or damage. A good strategy would entail replacing one server with another by powering the original server off and then powering it back on to restore service if the change was unsuccessful.
SEE: Checklist: How to manage your backups (TechRepublic Premium)
Note that some changes can’t be backed out by nature. An endeavor to replace a failed hard drive in a critical server is not going to lead to putting the failed drive back in the server, so set reasonable expections regarding back-out plans.
6. Establish timeframes for all tasks involved with the change, including the back-out plan
For instance, for a four-hour maintenance window, you might schedule two hours for the implementation plan, 30 minutes for the technical validation plan, 30 minutes for the business validation plan and one hour for the back-out plan. This helps ensure you do not cross the maintenance window threshold (but factor in what might happen if you do, just to be safe).
7. Get all stakeholders informed and on board with the change
Ensure all personnel who will be involved or impacted by the change are aware of the details and ramifications and have no concerns or objections (or if they do that these are satisfactorily addressed).
8. Conduct a thorough approval process
Approvals should involve managerial and executive personnel and should be based not only on authority to issue permission to proceed but technical know-how to confirm the change steps are sound. Approvals must take into account the risk assessment and change priority.
9. Follow the change management process via the approved change form
Close out all tasks as appropriate to mark them as complete or failed.
10. If applicable, back-out failed changes with a review and identification of what went wrong and why
Avoid engaging in blame games unless direct negligence was involved, and address such failures accordingly. Build a strategy to redo the change successfully.