Azure Monitor Container Insights for Azure Arc enabled Kubernetes clusters
Azure Monitor Container Insights provides rich monitoring experience for Azure Arc enabled Kubernetes clusters.
Azure Arc enabled Kubernetes preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. Azure Arc enabled Kubernetes previews are partially covered by customer support on a best-effort basis.
- Azure Monitor Container Insights supports monitoring Azure Arc enabled Kubernetes (preview) as described in the Overview article, except the live data (preview) feature. Also, users aren't required to have Owner permissions to enable metrics
Moby, and CRI compatible container runtimes such
- Outbound proxy without authentication and outbound proxy with basic authentication are supported. Outbound proxy that expects trusted certificates is currently not supported.
You've met the pre-requisites listed under the generic cluster extensions documentation.
A Log Analytics workspace: Azure Monitor Container Insights supports a Log Analytics workspace in the regions listed under Azure products by region page. You can create your own workspace through Azure Resource Manager, PowerShell, or Azure portal.
You need to have Contributor role assignment on the Azure subscription containing the Azure Arc enabled Kubernetes resource. If the Log Analytics workspace is in a different subscription, then Log Analytics Contributor role assignment is needed on the Log Analytics workspace.
To view the monitoring data, you need to have Log Analytics Reader role assignment on the Log Analytics workspace.
The following endpoints need to be enabled for outbound access in addition to the ones mentioned under connecting a Kubernetes cluster to Azure Arc.
If your Arc enabled Kubernetes resource is in Azure US Government environment, following endpoints need to be enabled for outbound access:
If you had previously deployed Azure Monitor Container Insights on this cluster using script without cluster extensions, follow the instructions listed here to delete this Helm chart. You can then continue to creating a cluster extension instance for Azure Monitor Container Insights.
The script-based version of deploying Azure Monitor Container Insights (preview) is being replaced by the cluster extension form of deployment. Azure Monitor deployed previously via script is only supported till June 2021 and it is thus advised to migrate to the cluster extension form of deployment at the earliest.
Identify workspace resource ID
Run the following commands to locate the full Azure Resource Manager identifier of the Log Analytics workspace.
List all the subscriptions that you have access to using the following command:
az account list --all -o table
Switch to the subscription hosting the Log Analytics workspace using the following command:
az account set -s <subscriptionId of the workspace>
The following example displays the list of workspaces in your subscriptions in the default JSON format.
az resource list --resource-type Microsoft.OperationalInsights/workspaces -o json
In the output, find the workspace name of interest. The
idfield of that represents the Azure Resource Manager identifier of that Log Analytics workspace.
idcan also be found in the Overview blade of the Log Analytics workspace through the Azure portal.
Create extension instance using Azure CLI
Option 1 - With default values
This option uses the following defaults:
- Creates or uses existing default log analytics workspace corresponding to the region of the cluster
- Auto-upgrade is enabled for the Azure Monitor cluster extension
az k8s-extension create --name azuremonitor-containers --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type connectedClusters --extension-type Microsoft.AzureMonitor.Containers
Option 2 - With existing Azure Log Analytics workspace
You can use an existing Azure Log Analytics workspace in any subscription on which you have Contributor or a more permissive role assignment.
az k8s-extension create --name azuremonitor-containers --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type connectedClusters --extension-type Microsoft.AzureMonitor.Containers --configuration-settings logAnalyticsWorkspaceResourceID=<armResourceIdOfExistingWorkspace>
Option 3 - With advanced configuration
If you want to tweak the default resource requests and limits, you can use the advanced configurations settings:
az k8s-extension create --name azuremonitor-containers --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type connectedClusters --extension-type Microsoft.AzureMonitor.Containers --configuration-settings omsagent.resources.daemonset.limits.cpu=150m omsagent.resources.daemonset.limits.memory=600Mi omsagent.resources.deployment.limits.cpu=1 omsagent.resources.deployment.limits.memory=750Mi
Checkout the resource requests and limits section of Helm chart for the available configuration settings.
Option 4 - On Azure Stack Edge
If the Azure Arc enabled Kubernetes cluster is on Azure Stack Edge, then a custom mount path
/home/data/docker needs to be used.
az k8s-extension create --name azuremonitor-containers --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type connectedClusters --extension-type Microsoft.AzureMonitor.Containers --configuration-settings omsagent.logsettings.custommountpath=/home/data/docker
If you are explicitly specifying the version of the extension to be installed in the create command, then ensure that the version specified is >= 2.8.2.
Create extension instance using Azure portal
If you are deploying Azure Monitor on a Kubernetes cluster running on top of Azure Stack Edge, then the Azure CLI option needs to be followed instead of the Azure portal option as a custom mount path needs to be set for these clusters.
Onboarding from the Azure Arc enabled Kubernetes resource blade
In the Azure portal, select the Arc enabled Kubernetes cluster that you wish to monitor.
Select the 'Insights (preview)' item under the 'Monitoring' section of the resource blade.
On the onboarding page, select the 'Configure Azure Monitor' button
You can now choose the Log Analytics workspace to send your metrics and logs data to.
Select the 'Configure' button to deploy the Azure Monitor Container Insights cluster extension.
Onboarding from Azure Monitor blade
In the Azure portal, navigate to the 'Monitor' blade, and select the 'Containers' option under the 'Insights' menu.
Select the 'Unmonitored clusters' tab to view the Azure Arc enabled Kubernetes clusters that you can enable monitoring for.
Click on the 'Enable' link next to the cluster that you want to enable monitoring for.
Choose the Log Analytics workspace and select the 'Configure' button to continue.
Create extension instance using Azure Resource Manager
Download Azure Resource Manager template and parameter:
curl -L https://aka.ms/arc-k8s-azmon-extension-arm-template -o arc-k8s-azmon-extension-arm-template.json curl -L https://aka.ms/arc-k8s-azmon-extension-arm-template-params -o arc-k8s-azmon-extension-arm-template-params.json
Update parameter values in arc-k8s-azmon-extension-arm-template-params.json file.For Azure public cloud,
opinsights.azure.comneeds to be used as the value of workspaceDomain.
Deploy the template to create Azure Monitor Container Insights extension
az login az account set --subscription "Subscription Name" az deployment group create --resource-group <resource-group> --template-file ./arc-k8s-azmon-extension-arm-template.json --parameters @./arc-k8s-azmon-extension-arm-template-params.json
Delete extension instance
The following command only deletes the extension instance, but doesn't delete the Log Analytics workspace. The data within the Log Analytics resource is left intact.
az k8s-extension delete --name azuremonitor-containers --cluster-type connectedClusters --cluster-name <cluster-name> --resource-group <resource-group>
If your cluster is disconnected from Azure for > 48 hours, then Azure Resource Graph won't have information about your cluster. As a result the Insights blade may display incorrect information about your cluster state.
With monitoring enabled to collect health and resource utilization of your Arc-enabled Kubernetes cluster and workloads running on them, learn how to use Container insights.
By default, the containerized agent collects the stdout/ stderr container logs of all the containers running in all the namespaces except kube-system. To configure container log collection specific to particular namespace or namespaces, review Container Insights agent configuration to configure desired data collection settings to your ConfigMap configurations file.
To scrape and analyze Prometheus metrics from your cluster, review Configure Prometheus metrics scraping