question

Ben-9958 avatar image
1 Vote"
Ben-9958 asked PeterFlion-6975 answered

Random Disconnections on RDS 2016 - Loads of ESENT errors

hi,

recently I've been having loads of rds disconnections on different servers (but not all) where all the users are disconnected and unable to login, when trying to login via console or mstsc /admin they fail as well (console just refreshes and removed the password from the field over and over) the only thing within the logs I can seem to find is a load (sometimes around 100) ESENT errors with different error numbers 482 is a usual one and the errors seem to say webcachetiles and insufficient resources. kind of run out of ideas with this... anything to do with AV or MRT?

any thoughts/help?

windows-serverremote-desktop-services
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi everyone.

We also have disconnects on multiple machines with the esent errors. Needing restart to be able to work again. Starting as stated begin july 2021 .
Tried running without antivirus / dsim /restorehealth etc.. nothing helps to stop/prevent it from happening.

If anyone has a solution to this, please also inform us.

Peter Flion
System Engineer

<Removed for Privacy>



1 Vote 1 ·

Hello,
Same issue, we are also in Belgium.
No solution. We opened a case beginning of July at Microsoft support, no solution.
We have the issue on vmware and hyper-v VM.
All those servers are using RDS services. OS are 2016 and 2019 Std.
We have decided to stop updating our customer's servers.

0 Votes 0 ·

HI

1.Could you please refer below document and find disconnect reason code or other related logS then share them in here?

Windows RDP-Related Event Logs: Identification, Tracking, and Investigation
https://ponderthebits.com/2018/02/windows-rdp-related-event-logs-identification-tracking-and-investigation/


Remote Desktop Client Troubleshooting: Disconnect Codes and Reasons
https://social.technet.microsoft.com/wiki/contents/articles/37870.remote-desktop-client-troubleshooting-disconnect-codes-and-reasons.aspx

2.Can we remote access the issue server 2016 by using local administrator account?

3.What's difference between both normal RD session host server and issue RD session host server?
(1)We can enter below powershell command on both normal server and issue server then check if there is any difference about windows update patch.
get-hotfix

0 Votes 0 ·

Anyone had any luck with Microsoft yet?

We unfortunately haven't - nearly 2 months this has been ongoing for, our end users are getting super frustrated.

0 Votes 0 ·

II changed the size of the VM, from a D16s v3 to 2x D8s v3 and this appears to have solved my issue on one of my collections. I've now made changed to another RDS collection from D16s v3 to a D32s v3. The correlation here being a D16s. Now, my question is, how may on this thread have their VM's running on a D16s v3?

0 Votes 0 ·
ManagedTechMSP-3273 avatar image
0 Votes"
ManagedTechMSP-3273 answered

Hello,

We have the exact same issue. Real story though, it doesn't only appear on RDS. It appears when using the system directly as well, or so it appears. Get a whole bunch of different ESENT errors but mainly the same as below.

This is now happening on multiple of our client systems and I've literally exhausted every option possible.

Did you find a fix for this? @Ben-9958

taskhostw (42936,D,0) WebCacheLocal: An attempt to write to the file "C:\Users\username\AppData\Local\Microsoft\Windows\WebCache\WebCacheV01.dat" at offset 14188544 (0x0000000000d88000) for 32768 (0x00008000) bytes failed after 0.000 seconds with system error 1453 (0x000005ad): "Insufficient quota to complete the requested service. ". The write operation will fail with error -1011 (0xfffffc0d). If this error persists then the file may be damaged and may need to be restored from a previous backup.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

IT-Guy-9874 avatar image
0 Votes"
IT-Guy-9874 answered random452389-7815 commented

Hello,
We have exactly the same issue on Hyper-v 2019 and vmware servers running Server 2019 and 2016.
The logs are the same as you.

The first issue starts on July the 5th on one machine. Now we have 5 servers with this issue.
When we connect on the console sometimes we cannot start the taskmgr.exe: Insufficient system resources exist to complete the requested service.
The only solution is to reboot the server.
The issue appear every one or two days.
We made a comparaison of the windows update installed but we did not found the reason.

We have an opened ticket at microsoft support but they still do not find the solution.

Edit: I found another post about the same issue: https://www.reddit.com/r/sysadmin/comments/ouhni1/rds_random_disconnections_unable_to_login_via/

· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Around the same time as us. Something must have happened around this time that we are all missing .

Its getting frustrating our clients are getting pissed off with us as we just can't fix it. Tried everything.

0 Votes 0 ·
IT-Guy-9874 avatar image IT-Guy-9874 ManagedTechMSP-3273 ·

Sure.
Microsoft support asked us to enable a lot of dump reports, they are all empty when the crash happens.
The server always crash during the day. Never during the night or weekend. So, a memory issue is possible.

1 Vote 1 ·

we encounter the rdp disconnect problems (with the esent errors) on 2012 2016 and 2019 servers.
All servers are guests on vmware esx 6.7 hosts.
They all run Microsoft 365 Apps for enterprise (version 16.0.14326.20222) -> i specify this because around begin July there were also many fixes for office.

It all started beginning July (the time when microsoft released the emergency printnightmare fix.). This fix (KB5004945) i removed but it doesn't stop the problem.

Clients / end users are indeed getting frustrated and so are we.




0 Votes 0 ·

what we did is downgrade the office version, this makes the problem go away, the only problem is you can't update office anymore. dont' forget to disable automatisc updates in Office itself.

command to downgrade.
“C:\Program Files\Common Files\microsoft shared\ClickToRun\officec2rclient.exe" /update user updatetoversion=16.0.14026.20308

the version used in the command is depended on which update channel you are using, take an version before July.

Another solution that seem to work on Hyper-V hosts is too put your memory on static instead of dynamic, don't know how this will translate to VMware though.


Regards

1 Vote 1 ·

This afternoon we will change the default port of some servers and change the ip-address assgined.
We do see alot of brute force connection attempts.
Cedric-6230 => you are from belgium, you also host your servers at OHV ?

You can join me directly at pf@tat.be

0 Votes 0 ·

I have wondered if its something to do with the office version

0 Votes 0 ·
ManagedTechMSP-3273 avatar image
0 Votes"
ManagedTechMSP-3273 answered

I can agree with some of the above posts.
I haven't downgraded office. I've completely removed office using the Office Removal Tool from Microsoft. I've then installed the latest 32bit version - at this stage, the issue has not occured for 3 days. I don't want to get my hopes up entirely yet, but removing office entirely, and installing 32bit does seem to resolve the issue.

As others have mentioned, downgrading office to a version prior to june does seem to also fix the issue.

Good luck all, I'll let you all knwo if issue occurs again.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PeterFlion-6975 avatar image
0 Votes"
PeterFlion-6975 answered

I can confirm that using following version of office does not have the problem: passing our problematic servers to this office version resolved the disconnect / esent error problems .

<Configuration ID="a37bde6f-de3b-45a1-85e8-58059d8442f5">
<Add OfficeClientEdition="64" Channel="Current" Version="16.0.14026.20308" MigrateArch="TRUE">
<Product ID="O365ProPlusRetail">

Dont forget to put the policies for office 365 to disable automatic updating and hide update notifications.

So no more office updates till microsoft fixes the problem...

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.