question

KevinS-5204 avatar image
8 Votes"
KevinS-5204 asked dborchert-4804 answered

KB4601318 fails to update, fails at 24% Windows Server 2016

Hello,

I have 3 virtual servers that have this exact same issue where when I try to update KB4601318, it stalls at 24%. The servers are 2016 servers, I have disabled anti-virus, I have rebooted, followed instructions on this blog, added RAM and enabled and disabled firewall. I have updated 15 other servers in my environment successfully, running the same antivirus software, same kind of setup. The servers all have 2 or 4 processors, 8 GB or 16 GB of RAM and at least 200+ GB of available disk space. No errors in the event viewer that I can see.

I've let the update run on these machines for 2 hours and what happens is that it will install KB890830 and KB4601392 on all of these machines but not KB4601318 on any of them. It gets stuck at 24% and then eventually stops and reads "KB4601318 is available" "Updates are ready" and Download button is enabled. If I click it, it goes through the exact same cycle where it stalls at 24% again.

Any assistance would be appreciated.

windows-server-2016
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Same situation here, setting up brand new AD environment, ESXi... Both DC and term server both stuck at 24% since yesterday... New servers, no past issues, confused on this as well.

3 Votes 3 ·

Both an older 2016 Hyper-V VM and brand new 2016 Hyper-V VM unable to complete updates! Hope this is fixed soon. I doubt a server with no updates will last for long!

Brandon.

0 Votes 0 ·

So if we've already downloaded and installed the SSU we will need to install the CU using the workaround? Or should we wait until a fix is available?

0 Votes 0 ·
JohnStorbeck-1365 avatar image
0 Votes"
JohnStorbeck-1365 answered

In the fix to this problem before removing KB4601318 from the catalog they mentioned that you should disable Express Updates on your SCCM Software Update Config.

Was everyone who had this issue with installing the Feb Cumulative Update also have the Check box to use Express Updates ticked?

We deployed the 2016 SSU ahead of our normal patch window and are trying to decide if we should pull the Feb Cumulative Update for 2016.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DamonRodriguez-4554 avatar image
0 Votes"
DamonRodriguez-4554 answered KevinS-5204 commented

I just realized I posted in the wrong place. If we already downloaded and installed the SSU do we need to manually install the CU using the work around or should we wait for a fixed update?

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Normally, security updates don't cause things to go sideways but if I could do it all over again, I would have waited.

0 Votes 0 ·
JohnStorbeck-1365 avatar image
0 Votes"
JohnStorbeck-1365 answered

I have a Sev A call open with MS Support. What they are telling me and this is based on internal docs and discussions with the devices team is that the issue with SCCM aka SECM is if you have your Software Updates settings setup to allow Express Updates. If this is turned on and you have deployed the 2016 Feb SSU you get the problem. If you don't do Express Updates you should not see an issue with the Feb 2021 Cumulative Update installing.

We deploy SSUs ahead of our normal scheduled weekend maintenance windows so we have over 3000 systems that already have the Feb 2016 SSU for 2016 KB4601392 installed and are not going to pull the Feb 2021 CU for 2016 KB4601318 from the SUG and see what happens during the early windows to decide if we are going to halt that deployment.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

rpodric avatar image
1 Vote"
rpodric answered

"NEW 2/12/21
Important There is a Known Issue that halts the installation progress of the February 9, 2021 security update. To address this issue, we have released a new servicing stack update (SSU), KB5001078. You must install this new SSU before installing the February 9, 2021 security update."


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SusanBradleyGSEC avatar image
0 Votes"
SusanBradleyGSEC answered

Windows Server 2016 SSU – has been recalled and been replaced with KB5001078. This fixes the issue where the cumulative update got stuck at 24%.

source: https://docs.microsoft.com/en-us/windows/release-health/status-windows-10-1607-and-windows-server-2016#1559msgdesc

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KevinS-5204 avatar image
2 Votes"
KevinS-5204 answered PascalBiettron-6550 commented

I FINALLY got it working but it would NOT work automatically, it kept hanging up at 0% for me, which isn't a big deal but I just wanted to relay my experience.

So I had to perform the exact same manual procedure that so many have posted already :

  1. First, I downloaded the new SSU (THANK YOU SusanBradleyGESC!!!) https://www.catalog.update.microsoft.com/Search.aspx?q=kb5001078

  2. Next, I already had the KB4601318 but it's here : https://www.catalog.update.microsoft.com/Search.aspx?q=kb4601318

  3. Next I opened a command prompt (CMD)

  4. I performed a NET STOP WUAUSERV (that did not work on two of my servers so I did the following:)
    4a.) SC QUERYEX WUAUSERV (to obtain the PID)
    4b.) TASKKILL /F /PID xxxx (where XXXX is the PID from the command from step 4a)

  5. RENAME %windir%\SoftwareDistribution SoftwareDistribution.bak

  6. I ran the new SSU KB5001078 (this run took about 3-5 minutes)

  7. I immediately ran the Feb security update KB4601318 (this run took about 30-45 minutes)

  8. Reboot (this takes the normal 10 or 15 minutes)

I know this probably isn't the outcome most of us would have liked but at least it was able to run. Honestly, if I had to do this on ALL 50 or so of my servers, I would be absolutely LIVID!

One thing I didn't try (and I should have but I just wanted this to be over and done with already, I mean, I started this at 1 AM and it's almost 5:00 AM and I worked a full 8-5 friggin' day today! Anyway, I'm wondering what would have happened if I would have just renamed the SoftwareDistribution folder and let this month's update run naturally?

Honestly everyone, for me, I tried a bunch of combinations but, ultimately, I honestly think renaming the ENTIRE SoftwareDistribution folder did the trick for me because stopping and starting the WUAUSERV service, running the download locally, running the new SSU, NONE of that was working for me until I ultimately renamed that folder.

Just my 2 cents. Big shout out to everyone that threw out their experiences, thank you so much everybody!

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you so much for sharing your experience KevinS-5204. It resolved my issue.

0 Votes 0 ·

Thank you so much for sharing your experience KevinS-5204. It resolved my issue too :-)

0 Votes 0 ·
Buskeyl avatar image
0 Votes"
Buskeyl answered Buskeyl converted comment to answer

So,

I appreciate all the information. Nice to know I am not on my own here. I am not quite sure how to tell at exactly what percentage they are failing at without catching it at the console, but to me it seems to be at 99%. I have over 60 Windows 2016 Servers failing to install KB4601318.

None have had the old SSU installed,(KB4601392) all have the updated SSU installed (KB5001078), and most are still failing to install KB4601318. They report 100% install of the patch shutting down, reboot, then on start up they go into the "Getting Windows Ready" phase. Start reporting "Working on Updates" and then eventually get to 99% complete, sit there for a while, Then comes back with "We couldn't complete the updates, Undoing changes Don't turn off your computer." We do not have any special builds, and maintain all via standard WSUS.

I haven't tried the manual procedure yet, but there's no way I am doing them all manually, MSFT needs to get this right. Not sure why we are having to do their testing.

· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I'm seeing KB4601318 fail and the new SSU successfully installs prior to reboot. I believe KB4601318 will install correctly after the reboot from our testing.

0 Votes 0 ·

Let us know. I have rebooted these servers 3 or 4 times now... Fails every time.

0 Votes 0 ·

Our failure rate was under 2% though I could not verify your exact condition.

0 Votes 0 ·
Show more comments
Buskeyl avatar image
0 Votes"
Buskeyl answered Buskeyl commented

Interesting. We are an on-prem virtual environment, but I'd not think the update mechanism would matter. For the Test VM's we run and let the patches auto install on, Cumulative Update KB4601318 was released and installed successfully before either Service Stack Update KB4601392 & KB5001078.

On a Test system:

68264-image.png


So based on our admittedly basic patch testing we were good to go. However when we released all the pending patches, it looks like SSU KB4601392 never went out because it was superseded by SSU KB5001078, and unlike the test machine above, SSU KB5001078 went on before CU KB4601318, and that was all she wrote. KB4601318 fails to install across the board, and does so in slightly different and fairly ugly way, failing at 99% complete of the post patch reboot maintenance. Might have just been a coincidence, but we also lost a domain controller during this time as well. Hard to say it was not a result of the failed CU..

On a Prod System:

68321-image.png



Converting this to an Answer for more visibility


image.png (76.4 KiB)
image.png (54.7 KiB)
· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Yeah that's a worry. I may hold back on the Servicing Stack update till I have the CU out of the way.

0 Votes 0 ·
Buskeyl avatar image Buskeyl SvendPetersen-7610 ·

So we figured out our issue. It was totally self inflicted.. Here the whole story in case it catches anyone else by the short hairs..
Part 1:

So, when converting the 2016 servers from MBR to GPT partitions as required to enable a UEFI BIOS, which is required for Secure Boot, which is required for the Windows option "Credential Guard" which is required by the STIG.... (STIG is a government security configuration guide for those who don't know)
Microsoft provides a tool called MBR2GPT.exe which allows the conversion of the disk to a GPT partition without reinstalling the OS. This worked fine, but for the majority of the VM's that needed to be converted, MBR2GPT created a third partition on the disk, and set that as the UFI partition required for GPT.
This totally works, but we had an unanticipated consequence. Because the new partition is created at the end of the disk, it prevents us from growing the main OS disk, something we need to do regularly.

1 Vote 1 ·

Part 2:

I was able to use a popular open-source tool called GPARTED Live utility to shift the main partition (2) to the right by 100MB, and the copy the 100 mb EFI partition (3) from the end of the disk to the freshly opened spot between the recovery partition (1) and the main partition (2), and then delete the original EFI partition that was in the way. While a little labor intensive, this worked great as afterward the modified VM's booted just fine.. Check that box and move on the next challenge..
Fast forward a few weeks, the 1.7 GB February Cumulative Update will not install. To further confuse matters, there was a totally separate issue with the February CU which would cause it to stall at 24% installed if install was attempted before a corrected Servicing Stack Update was released. That is a totally separate and unrelated issue to this.

0 Votes 0 ·
Show more comments
ZabagaRobertKingofPrussia-5177 avatar image
1 Vote"
ZabagaRobertKingofPrussia-5177 answered

This is the final answer I received from Microsoft's Devices and Deployment Team:

This is a Servicing Stack Update (Released on : 2021-02-09)
KB4601392

If you have installed this update on any machine through any possible way, it will face the following issue :
After installing KB4601392, installation of Cumulative Update from Windows Update might not progress past 24%.

Resolution :
KB4601392 already has been removed and is no longer offered to any machine.

New Servicing Stack Update (Released on : 2021-02-12)
KB5001078 – This SSU has to be installed and there will be no issues.

Action to be done on the machine that already had installed the corrupted SSU (KB4601392) :

  1.  Restart the machine
    
  2.  Using the Windows Update Reset Article : https://docs.microsoft.com/en-us/windows/deployment/update/windows-update-resources#reset-windows-update-components-manually
    
  3.  Follow step number : 1 , 2 and 4a
    
  4.  Restart your machine.
    

Now, if your question is how can this step be followed on thousands of machines :
1. SCCM – It can be used to push the commands and reboot.
2. Group Policy (Directory Services) - Step-By-Step instructions to setup a scheduled task
• Open the group policy management console.
• Right-click your domain and then click "Create a GPO in this domain, and Link it here ..".
• Give a name to this newly created GPO and click OK.
• Right-click this newly created GPO and then click Edit.
• Expand computer or user configuration and then go to the following path:
• Preferences -> Control Panel Settings -> Scheduled Tasks
• Right-click on scheduled tasks and then click New -> Scheduled Task.
• In the new task properties dialog box, provide the details of this task including action (choose create), run (choose script file), arguments (if any), comments, credentials and check Enabled at the bottom.
• Open schedule tab and set the schedule for this task to run.
• Click Apply and then click OK.
• Execute the gpupdate on the command prompt to apply the changes.

  1.  Manually on the affected machines.
    

Now, if you need assistance in following the workaround through SCCM or Group Policy, let us know. We will engage that team on a collaboration task, to help you.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

dborchert-4804 avatar image
0 Votes"
dborchert-4804 answered dborchert-4804 commented

First, thanks everyone for all the helpful information on this. I recently set up an update schedule for all my servers and while researching the available updates for them, I came across this thread... THANKFULLY!! I found KB4601318 installed on my primary domain controller that holds all FSMO roles and was pending a restart. I was able to uninstall it, but I am wondering if there's any additional manual work needed before I reboot it?

Even though KB4601318 is no longer listed in the "View Installed Updates" section (after I uninstalled it), I still see it in "Update Status" as shown in the image below, so that's why I'm asking if any additional manual work is needed before I reboot this. I almost want to transfer all FSMO roles temporarily to my other [backup] domain controller just in case this domain controller gets stuck at 24%. :(

I also tried to uninstall KB4601392 but the "uninstall" button disappeared when I clicked on the update...

Thanks!

73098-kb4601318-ixney.png

73099-bad-update-unsure-what-to-do.png



· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KB4601392 was the problem one. If you have KB4601318 and KB5001078 installed, you should be good. You should not have needed to remove anything.

2 Votes 2 ·

Yep. Also the issue was with downloading. In your case you have installed the update just fine

1 Vote 1 ·

Doh! Thanks for the quick reply. Just wanted to make sure I can reboot and not be worried.

0 Votes 0 ·