question

DenisPayne-4809 avatar image
4 Votes"
DenisPayne-4809 asked ·

Windows Defender creating thousands of files

Since 28/04/2021 around 22:00, thousands of files started to be created in folder C:\ProgramData\Microsoft\Windows Defender\Scans\History\Store\ on one of my domain controllers.

There were over 200k files which caused that night's backup to take over 4hours rather then the normal 20minutes.
There are now well over 400k files.

Another member server is also affected by this, there are over 2million files in the same folder being:
C:\ProgramData\Microsoft\Windows Defender\Scans\History\Store\

Bother servers are running Windows Server 2016.
The files are 1-2KB.
Settings>Update&Security>Windows Defender settings are enabled.
Windows Defender GUI>History is empty for Quarantined, Allowed and All Detected items.
No Windows Defender scan is running.

Resource Monitor>Disk>Disk Activity shows the System process accessing these files, so I presume it is creating them.
System is also the owner of these files.

windows-serverwindows-server-2016windows-server-security
· 7
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Same here since Friday!
We have Sophos and datto RMM on the systems.
Any news now?

0 Votes 0 ·

We are also facing this issue as well. We are on Windows Defender Engine version 1.1.18100.5 with Sophos installed. My colleague came across this Reddit thread (https://www.reddit.com/r/sysadmin/comments/n0q8pc/help_windows_defender_real_time_protection/) dealing with the same issue. Looks like an update to the engine to bring it to version 1.1.18100.6 may have resolved it for a few of those people but I don't think it's publically available yet. I've had at least 10 servers already affected by this and all of the issues started on 4/28. Hooray for Mondays!

0 Votes 0 ·

Is there any solution to this issue. We are encountering the same issue on windows Server 2k12r2 machines. There are number of files getting created under C:\ProgramData\Microsoft\Microsoft Antimalware\Scans\History\Store.

Any suggestions or solutions plz?

Regards,
Fazal

0 Votes 0 ·

For now, turn off Defender's Realtime Protection and it will stop creating the files.

0 Votes 0 ·
FazalKhan avatar image FazalKhan DavidFosbenner-1768 ·

We have already turned off real time protection and scheduled scan through SCCM antimalware policy and still no use. Everything is disabled for Defender.

Regards,
Fazal

0 Votes 0 ·
Show more comments

Same problem in non-server scenario.

Windows 10 Pro 1909

updated from 1.1.18100.5 to 1.1.18100.6 and the files are still there...

0 Votes 0 ·
DenisPayne-4809 avatar image
0 Votes"
DenisPayne-4809 answered ·

All in 7xWS2016 servers all running Sophos were affected, across two of my clients.
Windows Defender was thus uninstalled from 3xservers with small C-Drives to prevent 0% free space issue.

Cause seems to of been a MSFT Windows Defender update for which a fix was released late last week.

Windows Defender has been re-installed on the 3xservers it was previously uninstalled from.
None of the 7xWS2016 servers are showing a repeat of the issue so assume MSFT fixed it with a Windows Defender update.

10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DSPatrick avatar image
0 Votes"
DSPatrick answered ·

Something here may help.
https://support.microsoft.com/en-us/windows/protection-history-f1e5fd95-09b4-46d1-b8c7-1059a1e09708

--please don't forget to Accept as answer if the reply is helpful--



· 1
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

WS 2016 so protection history is in the seperate Windows Defender GUI, not yet integrated.

There are no detections, there is no scan running, just hundreds of thousands of 1-2KB files being created.

0 Votes 0 ·
PaulMartin-1885 avatar image
0 Votes"
PaulMartin-1885 answered ·

Just to add that we have also seen this issue which looks to be across multiple servers.
An example server has the folder is almost 2 million files large with the majority created from 29th April

Server 2016
Windows Defender Versions

Antimalware Client: 4.18.2001.7
Engine Version: 1.1.18100.5
Antivirus Definitions: 1.337.307.0
Network inspection system engine version: 1.1.18100.5
Network inspection system definition versions: 1.337.307.0

EDIT: We're also running Sophos on the impacted machines, I've raised a ticket with Sophos to see if they can check their side too or re-create the issue
After checking some servers though it seemed to start after a definitions update for Windows Defender after the MpKslacab service was re-installed
Between impacted servers, the "Engine Version" of Windows Defender seems to be the only one that matches other impacted servers too

· 1
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I've got this issue on 2 servers with the following

Antimalware client version: 4.18.2103.7
Engine version: 1.1.18100.5
Antivirus definition: 1.337.615.0
Network inspection system engine version: 1.1.18100.5
Network inspection system definition version: 1.337.615.0

I've checked other servers in our estate and the Engine version is 1.1.18100.6, these don't seem to be affected.

As the folder structure for Defender is locked updating it via Windows Updates isn't working, I'm assuming the files can't be accessed. I'm going to try templating one of our broken servers, removing the Defender feature, rebooting and then adding the feature again to see if that fixes the issue.

0 Votes 0 ·
DavidFosbenner-1768 avatar image
0 Votes"
DavidFosbenner-1768 answered ·

Oh thank goodness someone else has this issue! I thought I was losing my mind.

Starting on 4/29, 2 of my file servers suddenly had zero disk space. I'm running Windows Server 2019, I have the same issue with the same Store folder. This folder had about 1 million files, all under 2K, all dated within the last 24 hours. The only way I could stop creation of the files was disabling Defender's real-time protection. The files took up about 4GB. I deleted them all.

Since the servers are virtual machines I added 10GB to each C: drive. Well, guess what? Tonight the disks were full again, this time with over 11GB and 4 million files!

I just opened a case with MS PPI Support. When a server has no disk space things stop working, so obviously this is urgent. For now I've disable real-time protection and deleted the files again.

This is insane! I haven't made any system changes since the last patch Tuesday. I don't know what MS did but this is definitely on them IMO.

· 1
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Aye, sounds to be the recent Windows Defender definition update maybe.
Hoping MSFT picks up on this post and likely others to resolve the issue in next update.

0 Votes 0 ·
AndreasSchweizerdivertogmbh-8979 avatar image
0 Votes"
AndreasSchweizerdivertogmbh-8979 answered ·

Same here on some 2016 servers.
Any news from MS?
We habe Sophso Endpoint and datto RMM.. do you have some similar?

· 1
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

We have Sophos Endpoint and Atera RMM.
We were hit by ransomeware several months back so have opened some expensive Sophos support monitoring ticket.
They advised nothing to do with Sophos.

0 Votes 0 ·
DavidFosbenner-1768 avatar image
0 Votes"
DavidFosbenner-1768 answered ·

Good morning. Yes, I also have Sophos. I didn't get anything from MS yet, but I also opened a ticket with Sophos, asking about Defender AV and Sophos both running on the same system. It's been my experience in the past that when I install AV, it disables the native Windows AV, but not in this case. Here's what Sophos support said: (my comments in [brackets]).

"To assist you with your query Sophos can run with Windows Defender but it's advisable not to run both for we might encounter a performance issue when they run at the same time.

You can check by running command prompt as administrator, and run the fltmc command to see what drives are available on the server. [If fltmc returns "WdFilter" then Defender is running.]

Kindly try to disable the Windows Defender in Manage Roles and Features, untick the Windows Defender feature. [Reboot] and run the fltmc command again to confirm [WdFilter is removed]."

I did this on one server and confirmed Defender AV is no longer installed. It hopefully will solve the issue for me, but it doesn't explain the cause or truly "fix" what is broken. I might leave Defender on one server and work with MS to find the issue.

· 1
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Sophos was very shady with me about it was ok to run Sophos Central Endpoint alongside Windows Defender.

I thus just uninstalled Windows Defender from the 3xaffected WS2016 servers then after a reboot no new ..\Scans\History\Store\\ files being created.

0 Votes 0 ·
JamesFairless-0939 avatar image
0 Votes"
JamesFairless-0939 answered ·

We are also having the same issue on a Server 2019 VM box running Sophos. First noticed due to a large change in back up length and time. over 1 million new and modified files created in C:\ProgramData\Microsoft\Windows Defender\Scans\History\Store.

Also noticed that CPU usage was pinned with Sophos and Windows defender both being the culprits, starting to look like updates to definitions has both bits of software fighting each other causing the creation of millions of files.

Come on MS we need a support update please.

10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DavidFosbenner-1768 avatar image
0 Votes"
DavidFosbenner-1768 answered ·

While I'm curious to know what caused this, I'm not holding my breath waiting for MS. They'll probably blame Sophos. I have about a dozen servers, 4 had this issue. I uninstalled Defender from 10 servers and left it on 2 servers for testing/troubleshooting purposes. There's no need for it if the Sophos protection is there, so I don't see myself reinstalling Defender even if this issue gets fixed.

10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AndrewBounds-3913 avatar image
0 Votes"
AndrewBounds-3913 answered ·

2021-05-03:

Problem is with the AMEngineVersion < 1.1.18100.6. New version is supposed to be deployed by Microsoft Thursday May 6th.



I have this issue on a Windows 2008 R2 server running SCEP and a 2016 server running Defender. Do not have Sophos on either.

 Windows 2008 R2:
 C:\ProgramData\Microsoft\Microsoft Antimalware\Scans\History\Store
 Windows 2016:
 C:\ProgramData\Microsoft\Windows Defender\Scans\History\Store
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

YuhanDeng-MSFT avatar image
0 Votes"
YuhanDeng-MSFT answered ·

Hi,
Based on your description, I did some research but got nothing. To resolve this issue, I would suggest that you contact Microsoft Customer Support and Services where more in-depth investigation can be done so that you would get a more satisfying explanation and solution to this issue. In addition, if the issue has been proved as system flaw, the consulting fee would be refund. You may find phone number for your region accordingly from the link below.
Global Customer Service phone numbers:
https://support.microsoft.com/en-us/help/13948/global-customer-service-phone-numbers

Thanks for your time.
Best regards,
Danny


If the Answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.


10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.