Use Azure Container Storage Preview with Azure Elastic SAN

Azure Container Storage is a cloud-based volume management, deployment, and orchestration service built natively for containers. This article shows you how to configure Azure Container Storage to use Azure Elastic SAN as back-end storage for your Kubernetes workloads. At the end, you'll have a pod that's using Elastic SAN as its storage.

Prerequisites

  • If you don't have an Azure subscription, create a free account before you begin.

  • This article requires the latest version (2.35.0 or later) of the Azure CLI. See How to install the Azure CLI. If you're using the Bash environment in Azure Cloud Shell, the latest version is already installed. If you plan to run the commands locally instead of in Azure Cloud Shell, be sure to run them with administrative privileges. For more information, see Get started with Azure Cloud Shell.

  • You'll need the Kubernetes command-line client, kubectl. It's already installed if you're using Azure Cloud Shell, or you can install it locally by running the az aks install-cli command.

Note

To use Azure Container Storage with Azure Elastic SAN, your AKS cluster should have a node pool of at least three general purpose VMs such as standard_d4s_v5 for the cluster nodes, each with a minimum of four virtual CPUs (vCPUs).

Limitations

The following features aren't currently supported when you use Azure Container Storage to deploy and orchestrate an Elastic SAN.

  • Volume snapshots
  • Storage pool expansion

Regional availability

Azure Container Storage is only available for a subset of Azure regions:

  • (Africa) South Africa North
  • (Asia Pacific) Australia East
  • (Asia Pacific) East Asia
  • (Asia Pacific) Japan East
  • (Asia Pacific) Korea Central
  • (Asia Pacific) Southeast Asia
  • (Asia Pacific) Central India
  • (Europe) France Central
  • (Europe) North Europe
  • (Europe) West Europe
  • (Europe) UK South
  • (Europe) Sweden Central
  • (Europe) Switzerland North
  • (Middle East) UAE North
  • (North America) East US
  • (North America) East US 2
  • (North America) West US
  • (North America) West US 2
  • (North America) West US 3
  • (North America) Central US
  • (North America) North Central US
  • (North America) South Central US
  • (North America) West Central US
  • (North America) Canada Central
  • (North America) Canada East
  • (South America) Brazil South

Create a storage pool

First, create a storage pool, which is a logical grouping of storage for your Kubernetes cluster, by defining it in a YAML manifest file.

If you enabled Azure Container Storage using az aks create or az aks update commands, you might already have a storage pool. Use kubectl get sp -n acstor to get the list of storage pools. If you have a storage pool already available that you want to use, you can skip this section and proceed to Display the available storage classes.

Follow these steps to create a storage pool with Azure Elastic SAN.

  1. Use your favorite text editor to create a YAML manifest file such as code acstor-storagepool.yaml.

  2. Paste in the following code. The storage pool name value can be whatever you want. Adjust storage to reflect the storage capacity you want in Gi or Ti, and save the file. Azure Elastic SAN doesn't currently support resizing storage pools.

    apiVersion: containerstorage.azure.com/v1
    kind: StoragePool
    metadata:
      name: managed
      namespace: acstor
    spec:
      poolType:
        elasticSan: {}
      resources:
        requests: {"storage": 1Ti}
    
  3. Apply the YAML manifest file to create the storage pool.

    kubectl apply -f acstor-storagepool.yaml 
    

    When storage pool creation is complete, you'll see a message like:

    storagepool.containerstorage.azure.com/managed created
    

    You can also run this command to check the status of the storage pool. Replace <storage-pool-name> with your storage pool name value. For this example, the value would be managed.

    kubectl describe sp <storage-pool-name> -n acstor
    

When the storage pool is created, Azure Container Storage will create a storage class on your behalf using the naming convention acstor-<storage-pool-name>. It will also create an Azure Elastic SAN resource.

Assign Contributor role to AKS managed identity on Azure Elastic SAN subscription

Next, you must assign the Contributor Azure RBAC built-in role to the AKS managed identity on your Azure Elastic SAN subscription. You'll need an Owner role for your Azure subscription in order to do this. If you don't have sufficient permissions, ask your admin to perform these steps.

  1. Sign in to the Azure portal.

  2. Select Subscriptions, and locate and select the subscription associated with the Azure Elastic SAN resource that Azure Container Storage created on your behalf. This will likely be the same subscription as the AKS cluster that Azure Container Storage is installed on. You can verify this by locating the Elastic SAN resource in the resource group that AKS created (MC_YourResourceGroup_YourAKSClusterName_Region).

  3. Select Access control (IAM) from the left pane.

  4. Select Add > Add role assignment.

  5. Under Assignment type, select Privileged administrator roles and then Contributor, then select Next. If you don't have an Owner role on the subscription, you won't be able to add the Contributor role.

    Screenshot showing how to use the Azure portal to add Contributor role to the AKS managed identity.

  6. Under Assign access to, select Managed identity.

  7. Under Members, click + Select members. The Select managed identities menu will appear.

  8. Under Managed identity, select User-assigned managed identity.

  9. Under Select, search for and select the managed identity with your cluster name and -agentpool appended.

  10. Click Select, then Review + assign.

Display the available storage classes

When the storage pool is ready to use, you must select a storage class to define how storage is dynamically created when creating persistent volume claims and deploying persistent volumes.

Run kubectl get sc to display the available storage classes. You should see a storage class called acstor-<storage-pool-name>.

Important

Don't use the storage class that's marked internal. It's an internal storage class that's needed for Azure Container Storage to work.

Create a persistent volume claim

A persistent volume claim (PVC) is used to automatically provision storage based on a storage class. Follow these steps to create a PVC using the new storage class.

  1. Use your favorite text editor to create a YAML manifest file such as code acstor-pvc.yaml.

  2. Paste in the following code and save the file. The PVC name value can be whatever you want.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: managedpvc
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: acstor-managed # replace with the name of your storage class if different
      resources:
        requests:
          storage: 100Gi
    
  3. Apply the YAML manifest file to create the PVC.

    kubectl apply -f acstor-pvc.yaml
    

    You should see output similar to:

    persistentvolumeclaim/managedpvc created
    

    You can verify the status of the PVC by running the following command:

    kubectl describe pvc managedpvc
    

Once the PVC is created, it's ready for use by a pod.

Deploy a pod and attach a persistent volume

Create a pod using Fio (Flexible I/O Tester) for benchmarking and workload simulation, and specify a mount path for the persistent volume. For claimName, use the name value that you used when creating the persistent volume claim.

  1. Use your favorite text editor to create a YAML manifest file such as code acstor-pod.yaml.

  2. Paste in the following code and save the file.

    kind: Pod
    apiVersion: v1
    metadata:
      name: fiopod
    spec:
      nodeSelector:
        acstor.azure.com/io-engine: acstor
      volumes:
        - name: managedpv
          persistentVolumeClaim:
            claimName: managedpvc
      containers:
        - name: fio
          image: nixery.dev/shell/fio
          args:
            - sleep
            - "1000000"
          volumeMounts:
            - mountPath: "/volume"
              name: managedpv
    
  3. Apply the YAML manifest file to deploy the pod.

    kubectl apply -f acstor-pod.yaml
    

    You should see output similar to the following:

    pod/fiopod created
    
  4. Check that the pod is running and that the persistent volume claim has been bound successfully to the pod:

    kubectl describe pod fiopod
    kubectl describe pvc managedpvc
    
  5. Check fio testing to see its current status:

    kubectl exec -it fiopod -- fio --name=benchtest --size=800m --filename=/volume/test --direct=1 --rw=randrw --ioengine=libaio --bs=4k --iodepth=16 --numjobs=8 --time_based --runtime=60
    

You've now deployed a pod that's using an Elastic SAN as its storage, and you can use it for your Kubernetes workloads.

Detach and reattach a persistent volume

To detach a persistent volume, delete the pod that the persistent volume is attached to. Replace <pod-name> with the name of the pod, for example fiopod.

kubectl delete pods <pod-name>

To reattach a persistent volume, simply reference the persistent volume claim name in the YAML manifest file as described in Deploy a pod and attach a persistent volume.

To check which persistent volume a persistent volume claim is bound to, run kubectl get pvc <persistent-volume-claim-name>.

Delete a storage pool

If you want to delete a storage pool, run the following command. Replace <storage-pool-name> with the storage pool name.

kubectl delete sp -n acstor <storage-pool-name>

See also