Syslog collection with Container Insights
Container Insights offers the ability to collect Syslog events from Linux nodes in your Azure Kubernetes Service (AKS) clusters. This includes the ability to collect logs from control plane components like kubelet. Customers can also use Syslog for monitoring security and health events, typically by ingesting syslog into a SIEM system like Microsoft Sentinel.
Prerequisites
- You need to have managed identity authentication enabled on your cluster. To enable, see migrate your AKS cluster to managed identity authentication. Note: Enabling Managed Identity will create a new Data Collection Rule (DCR) named
MSCI-<WorkspaceRegion>-<ClusterName>
- Port 28330 should be available on the host node.
- Minimum versions of Azure components
- Azure CLI: Minimum version required for Azure CLI is 2.45.0 (link to release notes). See How to update the Azure CLI for upgrade instructions.
- Azure CLI AKS-Preview Extension: Minimum version required for AKS-Preview Azure CLI extension is 0.5.125 (link to release notes). See How to update extensions for upgrade guidance.
- Linux image version: Minimum version for AKS node linux image is 2022.11.01. See Upgrade Azure Kubernetes Service (AKS) node images for upgrade help.
How to enable Syslog
From the Azure portal
Navigate to your cluster. Open the Insights tab for your cluster. Open the Monitor Settings panel. Click on Edit collection settings, then check the box for Enable Syslog collection
Using Azure CLI commands
Use the following command in Azure CLI to enable syslog collection when you create a new AKS cluster.
az aks create -g syslog-rg -n new-cluster --enable-managed-identity --node-count 1 --enable-addons monitoring --enable-msi-auth-for-monitoring --enable-syslog --generate-ssh-key
Use the following command in Azure CLI to enable syslog collection on an existing AKS cluster.
az aks enable-addons -a monitoring --enable-msi-auth-for-monitoring --enable-syslog -g syslog-rg -n existing-cluster
Using ARM templates
You can also use ARM templates for enabling syslog collection
Download the template in the GitHub content file and save it as existingClusterOnboarding.json.
Download the parameter file in the GitHub content file and save it as existingClusterParam.json.
Edit the values in the parameter file:
aksResourceId
: Use the values on the AKS Overview page for the AKS cluster.aksResourceLocation
: Use the values on the AKS Overview page for the AKS cluster.workspaceResourceId
: Use the resource ID of your Log Analytics workspace.resourceTagValues
: Match the existing tag values specified for the existing Container insights extension data collection rule (DCR) of the cluster and the name of the DCR. The name will be MSCI-<clusterName>-<clusterRegion> and this resource created in an AKS clusters resource group. If this is the first time onboarding, you can set the arbitrary tag values.enableSyslog
: Set to truesyslogLevels
: Array of syslog levels to collect. Default collects all levels.syslogFacilities
: Array of syslog facilities to collect. Default collects all facilities
Note
Syslog level and facilities customization is currently only available via ARM templates.
Deploy the template
Deploy the template with the parameter file by using any valid method for deploying Resource Manager templates. For examples of different methods, see Deploy the sample templates.
Deploy with Azure PowerShell
New-AzResourceGroupDeployment -Name OnboardCluster -ResourceGroupName <ResourceGroupName> -TemplateFile .\existingClusterOnboarding.json -TemplateParameterFile .\existingClusterParam.json
The configuration change can take a few minutes to complete. When it's finished, a message similar to the following example includes this result:
provisioningState : Succeeded
Deploy with Azure CLI
az login
az account set --subscription "Subscription Name"
az deployment group create --resource-group <ResourceGroupName> --template-file ./existingClusterOnboarding.json --parameters @./existingClusterParam.json
The configuration change can take a few minutes to complete. When it's finished, a message similar to the following example includes this result:
provisioningState : Succeeded
How to access Syslog data
Access using built-in workbooks
To get a quick snapshot of your syslog data, customers can use our built-in Syslog workbook. There are two ways to access the built-in workbook.
Option 1 - The Reports tab in Container Insights. Navigate to your cluster. Open the Insights tab for your cluster. Open the Reports tab and look for the Syslog workbook.
Option 2 - The Workbooks tab in AKS Navigate to your cluster. Open the Workbooks tab for your cluster and look for the Syslog workbook.
Access using a Grafana dashboard
Customers can use our Syslog dashboard for Grafana to get an overview of their Syslog data. Customers who create a new Azure-managed Grafana instance will have this dashboard available by default. Customers with existing instances or those running their own instance can import the Syslog dashboard from the Grafana marketplace.
Note
You will need to have the Monitoring Reader role on the Subscription containing the Azure Managed Grafana instance to access syslog from Container Insights.
Access using log queries
Syslog data is stored in the Syslog table in your Log Analytics workspace. You can create your own log queries in Log Analytics to analyze this data or use any of the prebuilt queries.
You can open Log Analytics from the Logs menu in the Monitor menu to access Syslog data for all clusters or from the AKs cluster's menu to access Syslog data for only that cluster.
Sample queries
The following table provides different examples of log queries that retrieve Syslog records.
Query | Description |
---|---|
Syslog |
All Syslogs |
Syslog | where SeverityLevel == "error" |
All Syslog records with severity of error |
Syslog | summarize AggregatedValue = count() by Computer |
Count of Syslog records by computer |
Syslog | summarize AggregatedValue = count() by Facility |
Count of Syslog records by facility |
Syslog | where ProcessName == "kubelet" |
All Syslog records from the kubelet process |
Syslog | where ProcessName == "kubelet" and SeverityLevel == "error" |
Syslog records from kubelet process with errors |
Editing your Syslog collection settings
To modify the configuration for your Syslog collection, you modify the data collection rule (DCR) that was created when you enabled it.
Select Data Collection Rules from the Monitor menu in the Azure portal.
Select your DCR and then View data sources. Select the Linux Syslog data source to view the Syslog collection details.
Note
A DCR is created automatically when you enable syslog. The DCR follows the naming convention MSCI-<WorkspaceRegion>-<ClusterName>
.
Select the minimum log level for each facility that you want to collect.
Next steps
Once setup customers can start sending Syslog data to the tools of their choice
Read more
Share your feedback for this feature here: https://forms.office.com/r/BBvCjjDLTS
Зворотний зв’язок
https://aka.ms/ContentUserFeedback.
Очікується незабаром: протягом 2024 року ми будемо припиняти використання механізму реєстрації проблем у GitHub для зворотного зв’язку щодо вмісту й замінювати його новою системою зворотного зв’язку. Докладніше:Надіслати й переглянути відгук про