Tutorial: Install Azure Container Storage Preview for use with Azure Kubernetes Service

Article
03/12/2024

Azure Container Storage is a cloud-based volume management, deployment, and orchestration service built natively for containers. In this tutorial, you'll create an Azure Kubernetes Service (AKS) cluster and install Azure Container Storage on the cluster. Alternatively, you can install Azure Container Storage using a QuickStart instead of following the manual steps in this tutorial.

If you don't have an Azure subscription, create a free account before you begin.

Create a resource group
Choose a data storage option and VM type
Create an AKS cluster
Connect to the cluster
Label the node pool
Assign Contributor role to AKS managed identity
Install Azure Container Storage extension

Prerequisites

If you don't have an Azure subscription, create a free account before you begin.
This article requires the latest version (2.35.0 or later) of the Azure CLI. See How to install the Azure CLI. If you're using the Bash environment in Azure Cloud Shell, the latest version is already installed. If you plan to run the commands locally instead of in Azure Cloud Shell, be sure to run them with administrative privileges. For more information, see Get started with Azure Cloud Shell.
You'll need the Kubernetes command-line client, kubectl. It's already installed if you're using Azure Cloud Shell, or you can install it locally by running the az aks install-cli command.

Note

If you already have an AKS cluster deployed, proceed to Connect to the cluster.

Getting started

Take note of your Azure subscription ID. We recommend using a subscription on which you have a Kubernetes contributor role if you want to use Azure Disks or Ephemeral Disk as data storage. If you want to use Azure Elastic SAN as data storage, you'll need an Owner role on the Azure subscription.
Launch Azure Cloud Shell, or if you're using a local installation, sign in to the Azure CLI by using the az login command.
If you're using Azure Cloud Shell, you might be prompted to mount storage. Select the Azure subscription where you want to create the storage account and select Create.

Set subscription context

Set your Azure subscription context using the az account set command. You can view the subscription IDs for all the subscriptions you have access to by running the az account list --output table command. Remember to replace <subscription-id> with your subscription ID.

az account set --subscription <subscription-id>

Register resource providers

The Microsoft.ContainerService and Microsoft.KubernetesConfiguration resource providers must be registered on your Azure subscription. To register these providers, run the following command:

az provider register --namespace Microsoft.ContainerService --wait 
az provider register --namespace Microsoft.KubernetesConfiguration --wait

To check if these providers are registered successfully, run the following command:

az provider list --query "[?namespace=='Microsoft.ContainerService'].registrationState"
az provider list --query "[?namespace=='Microsoft.KubernetesConfiguration'].registrationState"

Create a resource group

An Azure resource group is a logical group that holds your Azure resources that you want to manage as a group. When you create a resource group, you're prompted to specify a location. This location is:

The storage location of your resource group metadata.
Where your resources will run in Azure if you don't specify another region during resource creation.

Create a resource group using the az group create command. Replace <resource-group-name> with the name of the resource group you want to create, and replace <location> with an Azure region such as eastus, westus2, westus3, or westeurope.

az group create --name <resource-group-name> --location <location>

If the resource group was created successfully, you'll see output similar to this:

{
  "id": "/subscriptions/<guid>/resourceGroups/myContainerStorageRG",
  "location": "eastus",
  "managedBy": null,
  "name": "myContainerStorageRG",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null
}

Choose a data storage option and virtual machine type

Before you create your cluster, you should understand which back-end storage option you'll ultimately choose to create your storage pool. This is because different storage services work best with different virtual machine (VM) types as cluster nodes, and you'll deploy your cluster before you create the storage pool.

Data storage options

Azure Elastic SAN: Azure Elastic SAN is a good fit for general purpose databases, streaming and messaging services, CD/CI environments, and other tier 1/tier 2 workloads. Storage is provisioned on demand per created volume and volume snapshot. Multiple clusters can access a single SAN concurrently, however persistent volumes can only be attached by one consumer at a time.
Azure Disks: Azure Disks are a good fit for databases such as MySQL, MongoDB, and PostgreSQL. Storage is provisioned per target container storage pool size and maximum volume size.
Ephemeral Disk: This option uses local NVMe or temp SSD drives on the AKS nodes and is extremely latency sensitive (low sub-ms latency), so it's best for applications with no data durability requirement or with built-in data replication support such as Cassandra. AKS discovers the available ephemeral storage on AKS nodes and acquires the drives for volume deployment.

VM types

To use Azure Container Storage, you'll need a node pool of at least three Linux VMs. Each VM should have a minimum of four virtual CPUs (vCPUs). Azure Container Storage will consume one core for I/O processing on every VM the extension is deployed to.

If you intend to use Azure Elastic SAN or Azure Disks with Azure Container Storage, then you should choose a general purpose VM type such as standard_d4s_v5 for the cluster nodes.

If you intend to use Ephemeral Disk, choose a storage optimized VM type such as standard_l8s_v3.

Important

You must choose a VM type that supports Azure premium storage.

Create AKS cluster

Run the following command to create a Linux-based AKS cluster and enable a system-assigned managed identity. If you already have an AKS cluster you want to use, you can skip this step.

Replace <resource-group> with the name of the resource group you created, <cluster-name> with the name of the cluster you want to create, and <vm-type> with the VM type you selected in the previous step. In this example, we'll create a cluster with three nodes. Increase the --node-count if you want a larger cluster.

az aks create -g <resource-group> -n <cluster-name> --node-count 3 -s <vm-type> --generate-ssh-keys

The deployment will take a few minutes to complete.

Note

When you create an AKS cluster, AKS automatically creates a second resource group to store the AKS resources. This second resource group follows the naming convention MC_YourResourceGroup_YourAKSClusterName_Region. For more information, see Why are two resource groups created with AKS?.

Connect to the cluster

To connect to the cluster, use the Kubernetes command-line client, kubectl. It's already installed if you're using Azure Cloud Shell, or you can install it locally by running the az aks install-cli command.

Configure kubectl to connect to your cluster using the az aks get-credentials command. The following command:
- Downloads credentials and configures the Kubernetes CLI to use them.
- Uses ~/.kube/config, the default location for the Kubernetes configuration file. You can specify a different location for your Kubernetes configuration file using the --file argument.
```
az aks get-credentials --resource-group <resource-group> --name <cluster-name>
```
Verify the connection to your cluster using the kubectl get command. This command returns a list of the cluster nodes.
```
kubectl get nodes
```

The following output example shows the nodes in your cluster. Make sure the status for all nodes shows Ready:

NAME                                STATUS   ROLES   AGE   VERSION
aks-nodepool1-34832848-vmss000000   Ready    agent   80m   v1.25.6
aks-nodepool1-34832848-vmss000001   Ready    agent   80m   v1.25.6
aks-nodepool1-34832848-vmss000002   Ready    agent   80m   v1.25.6

Take note of the name of your node pool. In this example, it would be nodepool1.

Label the node pool

Next, you must update your node pool label to associate the node pool with the correct IO engine for Azure Container Storage.

Important

If you created your AKS cluster using the Azure portal: The cluster will likely have a user node pool and a system/agent node pool. Before you can install Azure Container Storage, you must update the user node pool label as described in this section. However, if your cluster consists of only a system node pool, which is the case with test/dev clusters created with the Azure portal, you'll need to first add a new user node pool and then label it. This is because when you create an AKS cluster using the Azure portal, a taint CriticalAddOnsOnly is added to the agent/system nodepool, which blocks installation of Azure Container Storage on the system node pool. This taint isn't added when an AKS cluster is created using Azure CLI.

Run the following command to update the node pool label. Remember to replace <resource-group> and <cluster-name> with your own values, and replace <nodepool-name> with the name of your node pool.

az aks nodepool update --resource-group <resource-group> --cluster-name <cluster-name> --name <nodepool-name> --labels acstor.azure.com/io-engine=acstor

You can verify that the node pool is correctly labeled by signing into the Azure portal and navigating to your AKS cluster. Go to Settings > Node pools, select your node pool, and under Taints and labels you should see Labels: acstor.azure.com/io-engine:acstor.

Assign Contributor role to AKS managed identity

Azure Container Service is a separate service from AKS, so you'll need to grant permissions to allow Azure Container Storage to provision storage for your cluster. Specifically, you must assign the Contributor Azure RBAC built-in role to the AKS managed identity. You can do this using the Azure portal or Azure CLI. You'll need an Owner role for your Azure subscription in order to do this. If you don't have sufficient permissions, ask your admin to perform these steps.

Azure portal
Azure CLI

Sign in to the Azure portal, and search for and select Kubernetes services.
Locate and select your AKS cluster. Select Settings > Properties from the left navigation.
Under Infrastructure resource group, you should see a link to the resource group that AKS created when you created the cluster. Select it.
Select Access control (IAM) from the left pane.
Select Add > Add role assignment.
Under Assignment type, select Privileged administrator roles and then Contributor, then select Next. If you don't have an Owner role on the subscription, you won't be able to add the Contributor role.
Under Assign access to, select Managed identity.
Under Members, click + Select members. The Select managed identities menu will appear.
Under Managed identity, select User-assigned managed identity.
Under Select, search for and select the managed identity with your cluster name and -agentpool appended.
Click Select, then Review + assign.

Run the following commands to assign Contributor role to AKS managed identity. Remember to replace <resource-group>, <cluster-name>, and <azure-subscription-id> with your own values. You can also narrow the scope to your resource group, for example /subscriptions/<azure-subscription-id>/resourceGroups/<resource-group>.

export AKS_MI_OBJECT_ID=$(az aks show --name <cluster-name> --resource-group <resource-group> --query "identityProfile.kubeletidentity.objectId" -o tsv)
az role assignment create --assignee $AKS_MI_OBJECT_ID --role "Contributor" --scope "/subscriptions/<azure-subscription-id>"

Install Azure Container Storage

The initial install uses Azure Arc CLI commands to download a new extension. Replace <cluster-name> and <resource-group> with your own values. The <extension-name> value can be whatever you want; it's just a label for the extension you're installing.

During installation, you might be asked to install the k8s-extension. Select Y.

az k8s-extension create --cluster-type managedClusters --cluster-name <cluster-name> --resource-group <resource-group> --name <extension-name> --extension-type microsoft.azurecontainerstorage --scope cluster --release-train stable --release-namespace acstor

Installation takes 10-15 minutes to complete. You can check if the installation completed correctly by running the following command and ensuring that provisioningState says Succeeded:

az k8s-extension list --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type managedClusters

Congratulations, you've successfully installed Azure Container Storage. You now have new storage classes that you can use for your Kubernetes workloads.

Next steps

Now you can create a storage pool and persistent volume claim, and then deploy a pod and attach a persistent volume. Follow the steps in the appropriate how-to article.