Security

Manage patch deployment with these five steps


While recently working on a network with roughly 10,000 hosts, I discovered two machines that had gone unpatched for 11 months. This might not seem like a big deal -- .02 percent of machines that remained unpatched doesn't sound too bad.

But it only takes one unprotected machine to infect or disable an entire network. That means one unpatched workstation is a problem.

If you still question whether this really is an issue, here are some of the potential repercussions:

  • Downtime and loss of productivity due to reinstallation
  • Questionable data integrity due to a successful exploit
  • Negative public relations due to systems unavailable for your customers
  • Legal problems if your patch management process has to go under a judicial microscope

While patch deployment tools have become standardized fare on many midsize to large networks, there still remains a problem that many fail to notice or address: Some machines will always go unpatched.

No tool is going to solve this problem for you. Patching is a physical process -- not a tool. Some organizations treat patch deployment as a task. It's not a task; it's a full-time job.

Solving this problem takes several proactive steps. Here are five steps to managing the patch deployment process.

1. Inventory

Start by taking an inventory of your organization's network and the software deployed on it. Without this, you won't know which patches to deploy. In addition, prioritize machines by creating a risk profile based on their necessity to the organization.

2. Monitor

Once you have an inventory and prioritization of your assets, you're ready to start monitoring for patches. This is usually as simple as signing up for e-mail notification on the product vendor's Web site.

But don't rely on your vendor to always give you timely notification of problems. It may not notify you until it's developed a solution, and that could be too late. That's why you need to also monitor security Web sites for news of zero-day exploits and emerging problems.

3. Deploy

After you've set up a monitoring process, it's time to start deploying patches. There are many patch deployment solutions available, so pick one that fits your enterprise's needs and budget.

All solutions deliver patches. However, what you need most from your patch management system is a report on what the solution patched and which machines it did not patch.

4. Remediate

Now it's time to remediate. Using the report of which machines failed, track them down, and start monitoring them for activity.

Some users habitually turn off workstations at the end of the day regardless of how many e-mail notices they receive to leave them on. Monitor your network for activity from those machines.

In addition, you can use Wake-On-LAN (WOL) or Wake On Wireless LAN (WoWLAN). If you can't seem to catch the machine on or wake it up, take action.

5. Respond

Sometimes drastic action is necessary to protect your organization's network. You can always disable the machine's account and send a note to the help desk to route the trouble ticket through the patch management section.

If this doesn't work, then disable the switch port that the workstation connects to. Either solution will produce a response from the user and allow you to solve the short-term problem of deploying the patches. Then you can start working on the long-term problem of why this workstation is failing to receive patches.

Final thoughts

While automating patch management is possible, it's not always the best solution. It's important to inform management of the seriousness of such a situation and ask for the resources to do your job. The goal is zero failures -- it only takes one to bring down your entire network.

Mike Mullins has served as an assistant network administrator and a network security administrator for the U.S. Secret Service and the Defense Information Systems Agency. He is currently the director of operations for the Southern Theater Network Operations and Security Center.

Worried about security issues? Who isn't? Automatically sign up for our free Security Solutions newsletter, delivered each Friday, and get hands-on advice for locking down your systems.

10 comments
folsom6
folsom6

I think Mike could make it easier for Sir Scarfsalot to find him!! Try hot chocolate in the server! stevenandsandra@comcast.net

asgr86
asgr86

Can Somebody throw some light as to what kind of testing is required when a patch is deployed, for example on an DNS + ADS server what kind of testing are to be done and what are the parameters to be noted while testing. Thanks In advance.

nbalaj
nbalaj

Is there a documentation process? For example, is there a document, which can be given to the Engineer, who executes the patch installation process?

retro77
retro77

6. Automate The less you have to touch your end user systems the better. I am currently using WSUS 3.0 and it works.

Mike Mullins
Mike Mullins

Is one un-patched workstation/server to many or too few.

dkenny
dkenny

Other than support of Vista and new versions of Office, do you find the reasons to upgrade from WSUS 2.0 to 3.0 compelling? Or is this your first WSUS implementation in your organization?

bullens
bullens

Retro77 I agree WSUS 3.0 is fine, but it's only deploying patches to MS software, what about the rest of the products that you are using, i.e. Adobe Acrobat (Reader), winzip, to name a couple of common apps that you would expect to find on a standard deployment, how do you go about patching these?

Mond0
Mond0

Too few? What could possibly be the answer to that question? "We want more unpatched systems!"? Hahaha... I agree, whole-heartedly, with Mr. Brodock. I relish my horizontal time! Kudos, Mike.

michael.brodock
michael.brodock

managers seem to think that a couple (or more) workstations or servers that are unpatched is ok. Bad thinking in my opinion and experience. I can't really give details but having had to fight off viruses a couple of times still hasn't convinced some that one unpatched machine is too many. At least I get more money but I would rather just sleep well at night and be a pit poorer in pocket and richer in spirit. I really liked this article as well as most of your writings. Thanks and keep up the good work.

livedthere
livedthere

Use SMS to deploy the packages for patching, etc. Or look at Shavlik HFNetchk, thought it had additional functionality.