How to ingest large Recommendation files having size > 100 MB?

The below solution is not required anymore and this implementation is default now in the code. You should be able to ingest more than 100 MB with latest code changes.

Please implement this solution after consulting with a Microsoft Support Professional. There are inherent dangers associated with editing registry and manipulating the system.

If your Data collector machine generated Recommendation files having size >100 MB and you are not able to see data in Log Analytics, you can use below solution to ingest data into Log Analytics after meeting the Prerequisites.

Prerequisites:

  1. This solution is applicable to Windows Server, SCCM, SQL, Exchange and Active Directory Assessments only.

  2. This solution is available for the Data collector machines having bits version greater than or equal to 8.0.1.290.

    How to check bits version?

    Go to C:\Program Files\Microsoft Monitoring Agent\Agent\PowerShell\Microsoft.PowerShell.Oms.Assessments and check the version of the Microsoft.PowerShell.Oms.Assessments.dll version by right clicking and go to properties --> details tab.

What if you see a version less than 8.0.1.290?

  1. Open Services.msc and stop Microsoft Monitoring Agent service.
  2. Delete PowerShell and Health Service State folders from C:\Program Files\Microsoft Monitoring Agent\Agent location.

  1. Restart Microsoft Monitoring Agent service.
  2. Wait sometime to see PowerShell and Health Service State folders in C:\Program Files\Microsoft Monitoring Agent\Agent location.
  3. Check the bits version again. We should have latest version now.

After meeting the above Prerequisites, follow below steps.

Step 1: Create a registry string value under respective Assessment folder with the details below.

Key name - RecommendationFileRowCountPerBatch Value - 100000

Registry Folder Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HealthService\Parameters\Management Groups\AOI-WORKSPACEID\Solutions\ADAssessmentPlus

The screenshot below shows the registry key got added to Active Directory Assessment (ADAssessmentPlus)

Step 2: Re-run the Assessment by running the Task scheduler Task.

Below are the changes that can be noticed in the working directory after we do Step 1 and Step 2:

  1. Instead of 1 big Recommendation file having size >100 MB, you will see multiple new.* recommendation files having size <100 MB with Batch number in suffix as below.

  1. After new.* files got created, it takes approximately no.of new.* recommendation files x 20 minutes to change from new.* to processed.*.

    If we have 3 new.* recommendation files, it takes approximately 3 x 20 = 60 minutes to change files from files new.* to processed.*.

  2. Once these files renamed from new.* to processed.*, you can expect to see data in Log Analytics.

If you still don't see the data in Log Analytics, feel free to create a support ticket in Azure Portal.

How to undo the fix?

  1. Go ahead and delete the registry key "RecommendationFileRowCountPerBatch" under respective Assessment folder created above.

  1. After this, assessment will use the default process to ingest data into Log Analytics which works fine for ingesting files less than 100 MB.

For general feedback on the Resource Center or content, please submit your response to UserVoice. For specific requests and content updates regarding the Services Hub, contact our Support Team to submit a case.