Azure Policy Storage Account Diagnostic Logging

Question

question

SeanStark-9281 asked Apr 8, '21 VarunSharma-4299 edited May 9, '22

Azure Policy Storage Account Diagnostic Logging

I have been struggling to get this policy to fully check that blob, file, Queue, and Table services are properly configured with diagnostic settings. The end result would be to ensure each storage service has diagnostic logging being sent to a log analytics workspace for StorageRead, StorageWrite, and StorageDelete categories. To note this is using the newer method described in [https://docs.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage?tabs=azure-porta][1]l Also this could be an issue on the azure side, however it is unclear to me from this article if that is true or not. [https://github.com/azure/azure-policy#resource-type-not-correctly-published-by-resource-provider][2] The main issue I have been trying to solve is the logic to detect this across all blob services. The deployment logic works fine, its just the compliance check part to ensure blob, file, Queue, and Table services are enabled. Currently the below returns: (full policy attached as well) No related resources match the effect details in the policy definition. (Error code: Not Found) ["policyRule": { "if": { "field": "type", "equals": "Microsoft.Storage/storageAccounts" }, "then": { "effect": "[parameters('effect')]", "details": { "type": "Microsoft.Insights/diagnosticSettings", "existenceCondition": { "allOf": [ { "count": { "field": "Microsoft.Insights/diagnosticSettings/logs[]", "where": { "allOf": [ { "field": "Microsoft.Insights/diagnosticSettings/logs[].category", "in": [ "StorageRead", "StorageWrite", "StorageDelete" ] }, { "field": "Microsoft.Insights/diagnosticSettings/logs[*].enabled", "equals": "[parameters('logsEnabled')]" } ] } }, "greater": 0 }, { "field": "Microsoft.Insights/diagnosticSettings/workspaceId", "equals": "[parameters('logAnalytics')]" } ] },][3] [1]: https://docs.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage?tabs=azure-porta [2]: https://github.com/azure/azure-policy#resource-type-not-correctly-published-by-resource-provider [3]: /answers/storage/attachments/85864-storage-logging-policy.txt

azure-monitor azure-storage-accounts azure-policy

· 8

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SwathiDhanwada-MSFT · Apr 09, 2021 at 02:10 PM

anonymous user Welcome to Microsoft Q & A Community Forum. I speculate you might have checked built-in policy available to enable diagnostic settings for Azure Storage account. If not, I would suggest you check this built-in policy , if it fits your requirement. If not, kindly revert.

Reference Link : https://docs.microsoft.com/en-us/azure/storage/common/policy-reference?toc=/azure/storage/blobs/toc.json

0 ·

image.png (40.1 KiB)

SeanStark-9281 SwathiDhanwada-MSFT · Apr 09, 2021 at 03:47 PM

That built-in policy has the same issues, which is why I was trying the above. It does not report compliance properly on storage accounts that are configured correctly with diagnostic logging when the storage account category: metric (transaction) logging is not configured and underlying storage services such as blob, file, table, and queue are configured for logging.

As an example I configure the policy assignment and set the Transaction parameter as false. Indicating the desired state for the storage account is to have metric categories not enabled for logging and only the log categories (StorageDelete, StorageWrite, StorageRead) enabled for blob, queue, table, and file services.

In this scenario the policy assignment still shows these resources as non-compliant, even though the state is as intended. If and only when you configure metrics (transaction) logging does the policy show compliant regardless of the configured state of the blob, queue, table, and file services.

1 ·

SwathiDhanwada-MSFT SeanStark-9281 · Apr 16, 2021 at 02:36 PM

anonymous user Apologies for late response. Yes you were right, seems there is some discrepancy when evaluating the ARM properties. I am checking this issue with internal team and will update you soon.

0 ·

Show more comments

Answer 1 · 2022-05-09T16:38:57.717Z

VarunSharma-4299 answered May 9, '22 AnkitaRaniPatro-9753 commented May 9, '22

Yes I ended up creating custom policy with each different service type (blob, table, file, queue) to make it more manageable. The main change from the built-in policy is that the parameter "resourcename" in the deployment needs to be changed to refer to the full name "<storageAccountName>/default"

· 2

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AnkitaRaniPatro-9753 · May 09 at 04:43 PM

@VarunSharma-4299 ,Can we have a 5 min call today to guide me. I am completely stuck herev.Need to be done by today

0 ·

AnkitaRaniPatro-9753 · May 09 at 04:46 PM

@VarunSharma-4299 r u saying this one to change. ![200289-image.png][1] [1]: /answers/storage/attachments/200289-image.png

0 ·

Answer 2 · 2022-05-09T17:22:38.093Z

VarunSharma-4299 answered May 9, '22 VarunSharma-4299 edited May 9, '22

Yes the deployment needs to be updated to fullname (<storageAccountName>/default) rather than the service name (default)

"resourceName": {
"value": "[field('fullname')]"
}

and update the name in the deployment from

"name": "[concat(parameters('resourceName'), '/default/', 'Microsoft.Insights/', parameters('diagnosticsSettingNameToUse'))]",

to

"name": "[concat(parameters('resourceName'),'/', 'Microsoft.Insights/', parameters('diagnosticsSettingNameToUse'))]",

here is an e.g. for blob and you can modify it to use for others

{
"properties": {
"displayName": "Configure diagnostic settings for blob services",
"policyType": "Custom",
"mode": "All",
"description": "Deploys the diagnostic settings for storage account blob services to stream resource logs to a Log Analytics workspace when any storage account blob service which is missing these diagnostic settings is created or updated.",
"metadata": {
"category": "Storage"
},
"parameters": {
"logAnalytics": {
"type": "String",
"metadata": {
"displayName": "Log Analytics workspace",
"description": "Specify the Log Analytics workspace the storage account should be connected to.",
"strongType": "omsWorkspace",
"assignPermissions": true
}
},
"diagnosticsSettingNameToUse": {
"type": "String",
"metadata": {
"displayName": "Setting name",
"description": "Name of the diagnostic settings."
},
"defaultValue": "storageAccountsDiagnosticsLogsToWorkspace"
},
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "Enable or disable the execution of the policy"
},
"allowedValues": [
"DeployIfNotExists",
"Disabled"
],
"defaultValue": "DeployIfNotExists"
},
"StorageDelete": {
"type": "String",
"metadata": {
"displayName": "StorageDelete - Enabled",
"description": "Whether to stream StorageDelete logs to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
},
"StorageWrite": {
"type": "String",
"metadata": {
"displayName": "StorageWrite - Enabled",
"description": "Whether to stream StorageWrite logs to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
},
"StorageRead": {
"type": "String",
"metadata": {
"displayName": "StorageRead - Enabled",
"description": "Whether to stream StorageRead logs to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
},
"Transaction": {
"type": "String",
"metadata": {
"displayName": "Transaction - Enabled",
"description": "Whether to stream Transaction logs to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
}
},
"policyRule": {
"if": {
"field": "type",
"equals": "Microsoft.Storage/storageAccounts/blobServices"
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"type": "Microsoft.Insights/diagnosticSettings",
"roleDefinitionIds": [
"/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
"/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
],
"existenceCondition": {
"allOf": [{
"anyOf": [{
"field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
"equals": "True"
},
{
"field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
"equals": "True"
}
]
},
{
"field": "Microsoft.Insights/diagnosticSettings/workspaceId",
"equals": "[parameters('logAnalytics')]"
}
]
},
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"diagnosticsSettingNameToUse": {
"type": "string"
},
"resourceName": {
"type": "string"
},
"logAnalytics": {
"type": "string"
},
"location": {
"type": "string"
},
"Transaction": {
"type": "string"
},
"StorageRead": {
"type": "string"
},
"StorageWrite": {
"type": "string"
},
"StorageDelete": {
"type": "string"
}
},
"variables": {},
"resources": [{
"type": "Microsoft.Storage/storageAccounts/blobServices/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(parameters('resourceName'),'/', 'Microsoft.Insights/', parameters('diagnosticsSettingNameToUse'))]",
"location": "[parameters('location')]",
"dependsOn": [],
"properties": {
"workspaceId": "[parameters('logAnalytics')]",
"metrics": [{
"category": "Transaction",
"enabled": "[parameters('Transaction')]",
"retentionPolicy": {
"days": 0,
"enabled": false
},
"timeGrain": null
}],
"logs": [{
"category": "StorageRead",
"enabled": "[parameters('StorageRead')]"
},
{
"category": "StorageWrite",
"enabled": "[parameters('StorageWrite')]"
},
{
"category": "StorageDelete",
"enabled": "[parameters('StorageDelete')]"
}
]
}
}],
"outputs": {}
},
"parameters": {
"diagnosticsSettingNameToUse": {
"value": "[parameters('diagnosticsSettingNameToUse')]"
},
"logAnalytics": {
"value": "[parameters('logAnalytics')]"
},
"location": {
"value": "[field('location')]"
},
"resourceName": {
"value": "[field('fullname')]"
},
"Transaction": {
"value": "[parameters('Transaction')]"
},
"StorageDelete": {
"value": "[parameters('StorageDelete')]"
},
"StorageWrite": {
"value": "[parameters('StorageWrite')]"
},
"StorageRead": {
"value": "[parameters('StorageRead')]"
}
}
}
}
}
}
}
}
}

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

question