The Equifax security breach has now put 143 million Americans at risk for identity theft and fraud. It was caused by a website application vulnerability bug in Apache Foundation open source software that was known as Apache Struts CVE-2017-5638.
Apache patched this software vulnerability on March 7, 2017, and the appropriate software modifications were made by March 10–but Equifax did not download the latest release of the software that included the patch. Instead, it continued to operate its systems in a security-exposed environment.
“Considering Equifax is one of the largest credit reporting agencies whose sole business relies on both credibility of data and securely handling the sensitive data of millions of consumers, it is fair to say that they should have patched it as soon as possible, not to exceed a week. A typical bank would have patched this critical vulnerability within a few days,” said Pravin Kothari, CEO of CipherCloud, in an interview with USA Today.
Kothari is almost right. Having been a CIO in banking, we would have focused on installing a patch like this in hours, not a week.
SEE: Network security policy (Tech Pro Research)
How did something this serious get overlooked?
The short answer is that it shouldn’t have. But the realistic answer is that CIOs, IT leaders and project managers continue to put tasks that are perceived as mundane, like software maintenance and version updates, at the bottom of their to do lists.
There are reasons for this.
First, no one gets any glory for performing software maintenance. You’re just cleaning up someone else’s mistakes. Because you do this job, the software continues to run smoothly–but because the software continues to run smoothly, no one ever notices it and you never get any recognition.
When performance reviews come around, you never have any “spotlight events” that make you shine–so your chances of career advancement and salary hikes are limited.
Second, it is easy to push aside software updates and patches because of the number of projects that are always being managed with tight deadlines. The IT workload has to give somewhere–and it is often in the unsung area of software maintenance.
Finally, IT dreads software updates and patches. Quickly-done patches often create new incompatibilities (and problems) with apps in production because the apps no longer run smoothly with the introduction of the new patch. This means more work for IT.
However, if there is anything to be learned from the Equifax breach, it is that a set of best practices should be implemented in IT for software maintenance and critical patches.
Here are eight best practices:
1. Give some visibility to software updates and maintenance by using metrics
In one mainframe shop, the CIO kept a running record of how many days the mainframe ran without any unscheduled downtime. The record was 30 years and running! This certainly made the CEO and the board feel pretty good-and it gave some recognition to the folks in the trenches who were responsible for keeping hardware and software maintained.
2. Reward for performance
If you have metrics for maintenance, you can measure performance. Your maintenance staff should not be neglected because they aren’t superstar software developers. Recognize and reward them in salary increases and promotions.
3. Attack software bugs aggressively
There are some non-critical bugs in software that can wait–but if you’re faced with a security vulnerability, it can’t wait. The work must be at the front of the line–coming before projects.
SEE: Essential reading for IT leaders: 10 books on cybersecurity (free PDF) (TechRepublic)
4. Standardize your software version release process
With the growth of mobile computing, there is now a plethora of devices that new software versions and patches must be installed for. A standard software release process should be written and enforced to ensure that all devices are updated at the same time.
5. Use a push methodology for software updates
I have never been a fan of push software updates because they take control of a device and force a user to stop work and wait for new software to successfully install itself. However, after Equifax, I have reconsidered. Given the criticality of timely software patches, it seems to make sense for IT to push down these patch updates to devices for mandatory updates–even if users have to wait for software installs and restarts before they can resume work.
6. Change IT culture
IT should view software maintenance like an airline views jet maintenance. You don’t let a plane leave the ground if you know that a bolt on an engine is a little worn. IT needs to change its culture so that critical software patches aren’t pushed aside or neglected.
7. Test patches after they are installed
Because of urgency, patches are quickly installed and rapidly tested by the person patching the software. They may not go through QA at all. However, hastily installed patches can create software flaws of their own. After installation, new patches should be formally run through QA testing to ensure that patches are defect free.
8. Live with system incompatibilities
Sure, there’ll be patches to systems that create incompatibilities with your other production applications because a patch somehow interferes–but IT must learn to live with a certain amount of system incompatibility and inconvenience when the patch is critical. System incompatibility headaches are preferable to leaving millions of your customers at risk.
- 7 tips for effectively rolling out emergency patches (TechRepublic)
- 5 ways to make sure users comply with patch releases (TechRepublic)
- Why the Equifax breach could force executives to finally take cybersecurity seriously (TechRepublic)
- Information security incident reporting policy (Tech Pro Research)
- Equifax exposes credit services’ woeful IT, processes, security (ZDNet)
- Information Security Management Fundamentals (TechRepublic Academy)