question

NaveenKumarJainS-4926 avatar image
0 Votes"
NaveenKumarJainS-4926 asked tbgangav-MSFT commented

Log segregation capability in log analytics workspace

Hi,

I'm collecting the AKS container logs (application logs) in log analytics workspace. Does log analytics workspace is capable of segregating the logs as soon as they arrive in log analytics workspace based on a field present in log entry (like whether "type" in log entry is error or info) and route the log entry into a storage blob.

If Log analytics workspace is not capable of it, what is the best way to achieve this

azure-monitorazure-kubernetes-service
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

learn2skills avatar image
0 Votes"
learn2skills answered tbgangav-MSFT commented

Hi @NaveenKumarJainS-4926

For AKS Log Analytics may be your log aggregator of choice, You can export your container logs into Log Analytics and you can deploy the Azure Monitor solution.
After your data is ingested by Azure Monitor, the data is kept logically separate on each component throughout the service. All data is tagged per workspace. This tagging persists throughout the data lifecycle, and it is enforced at each layer of the service. Your data is stored in a dedicated database in the storage cluster in the region you have selected.

Container insights is a feature in Azure Monitor that monitors the health and performance of managed Kubernetes clusters hosted on AKS in addition to other cluster configurations. Container insights provides interactive views and workbooks that analyze collected data for a variety of monitoring scenarios.
https://docs.microsoft.com/en-us/azure/aks/monitor-aks#configure-monitoring




If the Answer is helpful, please click Accept Answer and up-vote, so that it can help others in the community looking for help on similar topics.


· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @learn2skills,

Thanks for replying to my question. The usecase i'm thinking on is having a custom logging as a JSON structure as below
Example:
{ "type": "ERROR", "msg": "Duplicate record found"    }
{ "type": "INFO", "msg": "successfully added record"    }

For example, these are the logs of application which is running inside the AKS containers. So I want to store these logs in separate storage account blobs as soon they are logged in container insights (i.e azure log analytics workspace )

What will be the suitable approach to do do this usecase?




0 Votes 0 ·
learn2skills avatar image learn2skills NaveenKumarJainS-4926 ·

Hi,

To achieve your scenario you can create custom logs, Refer to the below article it may help you.
Create Custom Log Analytics logs with LogicApps, powershell. Or we can use a powershell based Azure Function.
https://www.cloudsma.com/2019/05/custom-log-analytics-logs-logicappsps/

Collect custom logs with Log Analytics agent in Azure Monitor
https://docs.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-custom-logs



1 Vote 1 ·
tbgangav-MSFT avatar image tbgangav-MSFT NaveenKumarJainS-4926 ·

Hi @NaveenKumarJainS-4926,

Did you get chance to review the above response? Let us know if you have any further queries with regards to it.

1 Vote 1 ·