Not all patching problems are created equal
Credit to Author: Susan Bradley| Date: Mon, 16 May 2022 09:00:00 -0700
It’s the third week of the month — the week we find out whether Microsoft acknowledges any side effects it’s investigating as part of the monthly patch-release process.
First, a bit of background. Microsoft has released patches for years. But they haven’t always been released on a schedule. In the early days, Microsoft would release updates any day of the week. Then in October 2003, Microsoft formalized the release of normal security updates on the second Tuesday of the month. Thus was born Patch Tuesday. (Note: depending on where you are in the world, Patch Tuesday may be a Patch Wednesday.) The following day, or in some cases, over the next week, users and admins report issues with updates — and Microsoft finally acknowledges that, yes, there are issues.
Herein lies the rub: not everyone will see the side effects acknowledged by Microsoft (and sometimes there are side effects Microsoft never acknowledges). Or some that occur might simply be a coincidence of the patching process. (I’ve often installed updates and the act of rebooting brought to light an underlying issue I didn’t know about.)
This month, I made an interesting discovery. There are actually two sources of documentation about issues arising from the latest updates. The first, called the Windows Health Release Dashboard, lists all of the supported products from Windows Server 2022 all the way back to Windows 7 and documents issues Microsoft is investigating and has fixed. This month, for example, Microsoft acknowledges issues with Server 2022 triggered on Active Directory Domain Controllers. As the company notes: “An issue has been found related to how the mapping of certificates to machine accounts is being handled by the domain controller.”
Not all active directory domain controllers are affected — just those that use device certificates. Microsoft will be rolling out changes in how certificates are handled; it plans to add auditing now and enforce more changes later. If you are in charge of an Active Directory Domain I recommend you review this KB article and review your event.
Interestingly enough, there is a second source that documents patch problems Microsoft may be investigating. However, this recap of known issues is only available if you have access to an E3 or E5 license. If so, and you have either Administrator rights or Support rights, you can go to the integrated dashboard inside your Microsoft 365 dashboard. It documents some of the side effects not noted in the public dashboard. For instance, this month’s Microsoft 365 Health release dashboard acknowledged two additional issues not noted in the public console.
First, it notes the issue with Remote Desktop Services Broker Connection role:
“We have received reports that after installing KB5005575 or later updates on Windows Server 2022 Standard Edition, Remote Desktop Services Connection Broker role and supporting services might be removed unexpectedly. We have expedited investigation and are working on a resolution. Note: Windows Server 2022 Datacenter edition and other versions of Windows Server are not affected by this issue.
“Workaround: If you are using Remote Desktop Connection Broker on Windows Server 2022 Standard edition, you can mitigate this issue by removing Remote Desktop Connection Broker, installing the latest security update, and then re-adding Remote Desktop Connection Broker.
“Next steps: We are working on a resolution and will provide an update in an upcoming release.”
Next, it documents this:
“We are receiving reports that the Snip & Sketch app might fail to capture a screenshot or might fail to open using the keyboard shortcut (Windows key+shift+S), after installing KB5010386 and later updates.
“Next steps: We are presently investigating and will provide an update when more information is available.”
I’m unsure why there is a difference between the items noted in the public health release dashboard and the Microsoft 365 Health release dashboard. But if you have access to the Microsoft 365 version, you should review the information there.
More and more, Microsoft is using a technology called “Known Issue Rollback.” If a problem is introduced by a non-security fix included in the Patch Tuesday updates, Microsoft can roll it back and fix it behind the scenes. Often in the health release dashboard, you will see a notice that an issue will be handled this way and if you’re not in a corporate domain, you may be urged to reboot your computer. In a domain, you can use group policy as a trigger. (An admx file is routinely published with guidance to trigger the rollback.) These rollbacks can’t be done if the problem is triggered by a security patch, however, because returning the update to its pre-security patch state would leave your system vulnerable.
For example, a recent update introduced an issue where “some apps using Direct3D 9 might have issues on certain GPUs.”
As Microsoft notes:
“After installing KB5012643, Windows devices using certain GPUs might have apps close unexpectedly or intermittent issues with some apps which use Direct3D 9. You might also receive an error in Event Log in Windows Logs/Applications with faulting module d3d9on12.dll and exception code 0xc0000094.
“Resolution: This issue is resolved using Known Issue Rollback (KIR). Please note that it might take up to 24 hours for the resolution to propagate automatically to consumer devices and non-managed business devices. Restarting your Windows device might help the resolution apply to your device faster. For enterprise-managed, devices that have installed an affected update and encountered this issue can resolve it by installing and configuring the special Group Policy listed below. For information on deploying and configuring these special Group Policies, please see How to use Group Policy to deploy a Known Issue Rollback.
“Group Policy downloads with Group Policy name:
Once again, not all computers will see this problem. It’s limited to certain computers with specific GPUs that are affected.
Bottom line: the next time you see stories about side effects caused by Patch Tuesday releases, don’t assume you’ll be affected. You may encounter no issues whatsoever. If you have the resources, I recommend setting up a test bed of sample machines so you can determine if you will. If you can’t do that, the key to recovery (and avoiding issues), is to ensure you have a backup of your computer and can restore it if necessary. The technology that ensures you can recover from ransomware is also the same technology that ensures you can recover from errant patching side effects.