Use Azure ultra disks on Azure Kubernetes Service (preview)

• 07/10/2020
• 5 minutes to read
• • m
  • z
  • p
  • V
  • a

Azure ultra disks offer high throughput, high IOPS, and consistent low latency disk storage for your stateful applications. One major benefit of ultra disks is the ability to dynamically change the performance of the SSD along with your workloads without the need to restart your agent nodes. Ultra disks are suited for data-intensive workloads.

Before you begin

This feature can only be set at cluster creation or node pool creation time.

Register the EnableUltraSSD preview feature

To create an AKS cluster or a node pool that can leverage Ultra disks, you must enable the EnableUltraSSD feature flag on your subscription.

Register the EnableUltraSSD feature flag using the az feature register command as shown in the following example:

az feature register --namespace "Microsoft.ContainerService" --name "EnableUltraSSD"

It takes a few minutes for the status to show Registered. You can check on the registration status using the az feature list command:

az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnableUltraSSD')].{Name:name,State:properties.state}"

When ready, refresh the registration of the Microsoft.ContainerService resource provider using the az provider register command:

az provider register --namespace Microsoft.ContainerService

Install aks-preview CLI extension

To create an AKS cluster or a node pool that can use Ultra Disks, you need the latest aks-preview CLI extension. Install the aks-preview Azure CLI extension using the az extension add command, or install any available updates using the az extension update command:

# Install the aks-preview extension
az extension add --name aks-preview

# Update the extension to make sure you have the latest version installed
az extension update --name aks-preview

Limitations

• See the Ultra disks GA scope and limitations
• The supported size range for a Ultra disks is between 100 and 1500

Create a new cluster that can use Ultra disks

Create an AKS cluster that is able to leverage Ultra Disks by using the following CLI commands. Use the --aks-custom-headers flag to set the EnableUltraSSD feature.

Create an Azure resource group:

# Create an Azure resource group
az group create --name myResourceGroup --location westus2

Create the AKS cluster with support for Ultra Disks.

# Create an AKS-managed Azure AD cluster
az aks create -g MyResourceGroup -n MyManagedCluster -l westus2 --node-vm-size Standard_L8s_v2 --zones 1 2 --node-count 2 --aks-custom-headers EnableUltraSSD=true

If you want to create clusters without ultra disk support, you can do so by omitting the custom --aks-custom-headers parameter.

Enable Ultra disks on an existing cluster

You can enable ultra disks on existing clusters by adding a new node pool to your cluster that support ultra disks. Configure a new node pool to use host-based encryption by using the --aks-custom-headers flag.

az aks nodepool add --name ultradisk --cluster-name myAKSCluster --resource-group myResourceGroup --node-vm-size Standard_L8s_v2 --zones 1 2 --node-count 2 --aks-custom-headers EnableUltraSSD=true

If you want to create new node pools without support for ultra disks, you can do so by omitting the custom --aks-custom-headers parameter.

Use ultra disks dynamically with a storage class

To use ultra disks in our deployments or stateful sets you can use a storage class for dynamic provisioning.

Create the storage class

A storage class is used to define how a unit of storage is dynamically created with a persistent volume. For more information on Kubernetes storage classes, see Kubernetes Storage Classes.

In this case, we'll create a storage class that references ultra disks. Create a file named azure-ultra-disk-sc.yaml, and copy in the following manifest.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: ultra-disk-sc
provisioner: kubernetes.io/azure-disk
volumeBindingMode: WaitForFirstConsumer # optional, but recommended if you want to wait until the pod that will use this disk is created 
parameters:
  skuname: UltraSSD_LRS
  kind: managed
  cachingmode: None
  diskIopsReadWrite: "2000"  # minimum value: 2 IOPS/GiB 
  diskMbpsReadWrite: "320"   # minimum value: 0.032/GiB

Create the storage class with the kubectl apply command and specify your azure-ultra-disk-sc.yaml file:

$ kubectl apply -f azure-ultra-disk-sc.yaml


storageclass.storage.k8s.io/ultra-disk-sc created

Create a persistent volume claim

A persistent volume claim (PVC) is used to automatically provision storage based on a storage class. In this case, a PVC can use the previously created storage class to create an ultra disk.

Create a file named azure-ultra-disk-pvc.yaml, and copy in the following manifest. The claim requests a disk named ultra-disk that is 1000 GB in size with ReadWriteOnce access. The ultra-disk-sc storage class is specified as the storage class.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ultra-disk
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: ultra-disk-sc
  resources:
    requests:
      storage: 1000Gi

Create the persistent volume claim with the kubectl apply command and specify your azure-ultra-disk-pvc.yaml file:

$ kubectl apply -f azure-ultra-disk-pvc.yaml

persistentvolumeclaim/ultra-disk created

Use the persistent volume

Once the persistent volume claim has been created and the disk successfully provisioned, a pod can be created with access to the disk. The following manifest creates a basic NGINX pod that uses the persistent volume claim named ultra-disk to mount the Azure disk at the path /mnt/azure.

Create a file named nginx-ultra.yaml, and copy in the following manifest.

kind: Pod
apiVersion: v1
metadata:
  name: nginx-ultra
spec:
  containers:
  - name: nginx-ultra
    image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi
    volumeMounts:
    - mountPath: "/mnt/azure"
      name: volume
  volumes:
    - name: volume
      persistentVolumeClaim:
        claimName: ultra-disk

Create the pod with the kubectl apply command, as shown in the following example:

$ kubectl apply -f nginx-ultra.yaml

pod/nginx-ultra created

You now have a running pod with your Azure disk mounted in the /mnt/azure directory. This configuration can be seen when inspecting your pod via kubectl describe pod nginx-ultra, as shown in the following condensed example:

$ kubectl describe pod nginx-ultra

[...]
Volumes:
  volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  azure-managed-disk
    ReadOnly:   false
  default-token-smm2n:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-smm2n
    Optional:    false
[...]
Events:
  Type    Reason                 Age   From                               Message
  ----    ------                 ----  ----                               -------
  Normal  Scheduled              2m    default-scheduler                  Successfully assigned mypod to aks-nodepool1-79590246-0
  Normal  SuccessfulMountVolume  2m    kubelet, aks-nodepool1-79590246-0  MountVolume.SetUp succeeded for volume "default-token-smm2n"
  Normal  SuccessfulMountVolume  1m    kubelet, aks-nodepool1-79590246-0  MountVolume.SetUp succeeded for volume "pvc-faf0f176-8b8d-11e8-923b-deb28c58d242"
[...]

Next steps

• For more about ultra disks, see Using Azure ultra disks.
• For more about storage best practices, see Best practices for storage and backups in Azure Kubernetes Service (AKS)