In a previous article here on TechGenix, I argued that Microsoft’s approach to patch management has reached a tipping point because the pain of patching Windows operating systems and applications has almost become unbearable for many administrators. I said that something has to give somewhere, and soon, otherwise those of us who have Windows deployed across our organizations are in big trouble.
But no one wants to be in trouble, so maybe there are some mitigating steps we can take as patch management admins to stave off impending doom. Those steps, of course, include both best practices for patch management as well as insider tips on how to deal with the mess Microsoft seems hell-bent on creating in this area. Hopefully, the steps described below will give you some breathing room until Microsoft realizes it has to stop using its customers as live testers for its software updates and retool its whole approach to testing and releasing updates for Windows and applications like Microsoft Office.
Wait a week before applying updates
Perhaps the first and most important thing you can do about patch management is to wait at least a week before applying a software update after it’s been released by Microsoft. This means, for example, that you should be ready to apply patches about a week after Patch Tuesday, the day when Microsoft releases new patches for Windows and which is the second (and sometimes the fourth) Tuesday of each month.
This delaying tactic should also be the case despite whatever end-of-the-world urgency Microsoft issues regarding the critical importance of the patch. Large organizations benefit from this by allowing themselves sufficient time to test the consequences of the update by applying it first in a test environment that closely mirrors their production environment. Small- and midsized businesses (SMBs) may not have the IT personnel resources needed to perform a thorough test of a patch before applying it, but a one week wait still benefits them as well since it can give them time to review both Microsoft and third-party resources that can help them determine whether applying the update might result in some a negative impact upon their environment. We’ll focus on some of these additional resources next.
Review the known issues Microsoft identifies for each patch
When Microsoft sends out an email notification of the release of a new security update, the email usually includes mention of any known issues that Microsoft has identified concerning the update. For example, a few weeks ago I received an email from Microsoft issued on August 1 with the title Microsoft Security Update Releases that said the following CVEs (Common Vulnerabilities and Exposures) had undergone a major revision increment: CVE-2018-8172 which involves a Visual Studio Remote Code Execution Vulnerability, and CVE-2018-8202 which involves a .NET Framework Elevation of Privilege Vulnerability. In other words, Microsoft had released an update to patch two earlier updates that had failed to completely resolve some previously identified vulnerabilities. In the email Microsoft said it was “announcing the release of updates, available via the Microsoft Update catalog, to resolve known issues some customers experienced after installing the July 2018 security updates for .NET Framework.”
Information about released software updates is also available in articles published on the Microsoft Support website. For example, an article titled “August 14, 2018—KB4343909 (OS Build 17134.228)” which applies to Windows 10 v.1803 announces updates for Microsoft HoloLens that provides some quality improvements. But the article also identifies some known issues concerning the update:
My point in mentioning all this is that if you’re in charge of patch management for your organization, then you and your team need to take time to read these notification emails carefully (or review the information on the support pages for each new update) and especially the Known Issues section as a heads-up on whether applying the patch might cause more problems for your environment than it solves. So don’t be lazy! Do your homework!
Review news on third-party patch monitoring sites
Unfortunately, smaller businesses may not have the time or expertise to fully review and understand the flood of information pouring out of Redmond each month describing the new patches they are issuing to patch earlier patches that failed to fix stuff properly. Fortunately, the overworked IT person for such organizations has recourse to some excellent tech news sites that focus on problems arising from customers applying new software updates. Woody Leonhard’s Ask Woody website is one of the best sources small business IT can tap into regarding what might blow up if they apply a newly released software update. And as of earlier this year Microsoft MVP Susan Bradley a.k.a. “The Patch Lady,” now regularly contributes to Woody’s site, and her articles can be invaluable for simplifying and clarifying Microsoft’s often confusing message concerning its software updates. Tom Walat, site editor for TechTarget, also frequently publishes valuable content in the area of what’s good and bad about the latest patches from Redmond.
Patch management: Speed up and slow down
Finally, it’s a good idea to modify your one-week delay of patching by both speeding it up and slowing it down. Specifically, for your most sensitive systems (key servers, for example) you may want to factor in an extra week or two to carefully test the potential impact that may occur from applying a software update. Microsoft Exchange admins and IIS admins are two examples of those who have been burned several times in the past from applying a new update too soon, resulting in mail systems going down and online storefronts throwing errors.
On the other hand, it’s also a good idea to apply the patch immediately to a few key systems so you don’t fall too far behind what the bad guys are doing — provided of course you have functionality in place that allows you to quickly roll back those systems should this be necessary. While deploying patches first in a test environment mirroring production is always a good idea, the reality is that most test environments end up lagging far behind production environments because users don’t do real work on test clients and admins don’t perform real administration of test servers. So while you’re busy wading through reams of security update emails from Microsoft and reading news items on Ask Woody and TechTarget, you should also take the plunge and patch a few machines and see if anything blows up.
And oh yeah — if your anxiety from patch management grows unbearable, there’s always Valium.
Featured image: Shutterstock
