question

jonathandhope avatar image
0 Votes"
jonathandhope asked ·

Size of storage account (tables) for Service Fabric Diagnostics

Has any guidance been published on best practices around collecting diagnostic logs for Service Fabric. Our Diagnostic storage accounts are currently over 4TB, the majority of which seems to be taken up by the WADServiceFabricSystemEventTable and WADPatchOrchestrationApplicationTable tables?

Only information I can find is here service-fabric-best-practices-monitoring.md, but this doesn't talk about retention.

Here is the diagnostics snippet from the ARM template we use to create the scalesets:

 "publisher": "Microsoft.Azure.Diagnostics",
                   "settings": {
                     "WadCfg": {
                       "DiagnosticMonitorConfiguration": {
                         "overallQuotaInMB": "50000",
                         "EtwProviders": {
                           "EtwEventSourceProviderConfiguration": [
                             {
                               "provider": "Microsoft-ServiceFabric-Actors",
                               "scheduledTransferKeywordFilter": "1",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "ServiceFabricReliableActorEventTable"
                               }
                             },
                             {
                               "provider": "Microsoft-ServiceFabric-Services",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "ServiceFabricReliableServiceEventTable"
                               }
                             },
                             {
                               "provider": "e39b723c-590c-4090-abb0-11e3e6616346",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "PatchOrchestrationApplicationTable"
                               }
                             },
                             {
                               "provider": "fc0028ff-bfdc-499f-80dc-ed922c52c5e9",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "PatchOrchestrationApplicationTable"
                               }
                             },
                             {
                               "provider": "24afa313-0d3b-4c7c-b485-1047fd964b60",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "PatchOrchestrationApplicationTable"
                               }
                             },
                             {
                               "provider": "05dc046c-60e9-4ef7-965e-91660adffa68",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "PatchOrchestrationApplicationTable"
                               }
                             },
                             {
                               "provider": "Asos.Finance.EtlLogs",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "AsosFinanceEtlLogs"
                               }
                             }
                           ],
                           "EtwManifestProviderConfiguration": [
                             {
                               "provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
                               "scheduledTransferLogLevelFilter": "Information",
                               "scheduledTransferKeywordFilter": "4611686018427387912",
                               "scheduledTransferPeriod": "PT5M",
                               "DefaultEvents": {
                                 "eventDestination": "ServiceFabricSystemEventTable"
                               }
                             }
                           ]
                         }
                       }
                     }


azure-service-fabric
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

jonathandhope avatar image
1 Vote"
jonathandhope answered ·

I'd missed this section from the guidance:

[!NOTE] There is currently no way to filter or groom the events that are sent to the table. If you don't implement a process to remove events from the table, the table will continue to grow. Currently, there is an example of a data grooming service running in the Watchdog sample, and it is recommended that you write one for yourself as well, unless there is a good reason for you to store logs beyond a 30 or 90 day timeframe.

Looks like we'll have to add to our Watchdog



·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.