Integrate Azure HPC Cache with Azure Kubernetes Service (AKS)

Article
02/16/2024

Azure HPC Cache speeds access to your data for high-performance computing (HPC) tasks. By caching files in Azure, Azure HPC Cache brings the scalability of cloud computing to your existing workflow. This article shows you how to integrate Azure HPC Cache with Azure Kubernetes Service (AKS).

Before you begin

AKS cluster must be in a region that supports Azure HPC Cache.
You need Azure CLI version 2.7 or later. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.
Register the hpc-cache extension in your Azure subscription. For more information on using HPC Cache with Azure CLI, see the HPC Cache CLI prerequisites.
Review the HPC Cache prerequisites. You need to satisfy the following before you can run an HPC Cache:
- The cache requires a dedicated subnet with at least 64 IP addresses available.
- The subnet must not host other VMs or containers.
- The subnet must be accessible from the AKS nodes.
If you need to run your application as a user without root access, you may need to disable root squashing by using the change owner (chown) command to change directory ownership to another user. The user without root access needs to own a directory to access the file system. For the user to own a directory, the root user must chown a directory to that user, but if the HPC Cache is squashing root, this operation is denied because the root user (UID 0) is being mapped to the anonymous user. For more information about root squashing and client access policies, see HPC Cache access policies.

Install the `hpc-cache` Azure CLI extension

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

To install the hpc-cache extension, run the following command:

az extension add --name hpc-cache

Run the following command to update to the latest version of the extension released:

az extension update --name hpc-cache

Register the StorageCache feature flag

az provider register --namespace Microsoft.StorageCache --wait

It takes a few minutes for the status to show Registered. Verify the registration status by using the az feature show command:

az feature show --namespace "Microsoft.StorageCache"

Create the Azure HPC Cache

Get the node resource group using the az aks show command with the --query nodeResourceGroup query parameter.
```
az aks show --resource-group myResourceGroup --name myAKSCluster --query nodeResourceGroup -o tsv
```
Your output should look similar to the following example output:
```
MC_myResourceGroup_myAKSCluster_eastus
```

Create a dedicated HPC Cache subnet using the az network vnet subnet create command. First define the environment variables for RESOURCE_GROUP, VNET_NAME, VNET_ID, and SUBNET_NAME. Copy the output from the previous step for RESOURCE_GROUP, and specify a value for SUBNET_NAME.

RESOURCE_GROUP=MC_myResourceGroup_myAKSCluster_eastus
VNET_NAME=$(az network vnet list --resource-group $RESOURCE_GROUP --query [].name -o tsv)
VNET_ID=$(az network vnet show --resource-group $RESOURCE_GROUP --name $VNET_NAME --query "id" -o tsv)
SUBNET_NAME=MyHpcCacheSubnet

az network vnet subnet create \
    --resource-group $RESOURCE_GROUP \
    --vnet-name $VNET_NAME \
    --name $SUBNET_NAME \
    --address-prefixes 10.0.0.0/26

Create an HPC Cache in the same node resource group and region. First define the environment variable SUBNET_ID.
```
SUBNET_ID=$(az network vnet subnet show --resource-group $RESOURCE_GROUP --vnet-name $VNET_NAME --name $SUBNET_NAME --query "id" -o tsv)
```
Create the HPC Cache using the az hpc-cache create command. The following example creates the HPC Cache in the East US region with a Standard 2G cache type named MyHpcCache. Specify a value for --location, --sku-name, and --name.
```
az hpc-cache create \
  --resource-group $RESOURCE_GROUP \
  --cache-size-gb "3072" \
  --location eastus \
  --subnet $SUBNET_ID \
  --sku-name "Standard_2G" \
  --name MyHpcCache
```
Note

Creation of the HPC Cache can take up to 20 minutes.

Create and configure Azure storage

Create a storage account using the az storage account create command. First define the environment variable STORAGE_ACCOUNT_NAME.

Important

You need to select a unique storage account name. Replace uniquestorageaccount with your specified name. Storage account names must be between 3 and 24 characters in length and can contain only numbers and lowercase letters.
```
STORAGE_ACCOUNT_NAME=uniquestorageaccount
```
The following example creates a storage account in the East US region with the Standard_LRS SKU. Specify a value for --location and --sku.
```
az storage account create \
  --name $STORAGE_ACCOUNT_NAME \
  --resource-group $RESOURCE_GROUP \
  --location eastus \
  --sku Standard_LRS
```

Assign the Storage Blob Data Contributor Role on your subscription using the az role assignment create command. First, define the environment variables STORAGE_ACCOUNT_ID and AD_USER.

STORAGE_ACCOUNT_ID=$(az storage account show --name $STORAGE_ACCOUNT_NAME --query "id" -o tsv)
AD_USER=$(az ad signed-in-user show --query objectId -o tsv)

az role assignment create --role "Storage Blob Data Contributor" --assignee $AD_USER --scope $STORAGE_ACCOUNT_ID

Create the Blob container within the storage account using the az storage container create command. First, define the environment variable CONTAINER_NAME and replace the name for the Blob container.
```
CONTAINER_NAME=mystoragecontainer
```
```
az storage container create --name $CONTAINER_NAME --account-name $STORAGE_ACCOUNT_NAME --auth-mode login
```

Provide permissions to the Azure HPC Cache service account to access your storage account and Blob container using the az role assignment commands. First, define the environment variables HPC_CACHE_USER and HPC_CACHE_ID.

HPC_CACHE_USER="StorageCache Resource Provider"
HPC_CACHE_ID=$(az ad sp list --display-name "${HPC_CACHE_USER}" --query "[].objectId" -o tsv)

az role assignment create --role "Storage Account Contributor" --assignee $HPC_CACHE_ID --scope $STORAGE_ACCOUNT_ID
az role assignment create --role "Storage Blob Data Contributor" --assignee $HPC_CACHE_ID --scope $STORAGE_ACCOUNT_ID

Add the blob container to your HPC Cache as a storage target using the az hpc-cache blob-storage-target add command. The following example creates a blob container named MyStorageTarget to the HPC Cache MyHpcCache. Specify a value for --name, --cache-name, and --virtual-namespace-path.
```
az hpc-cache blob-storage-target add \
  --resource-group $RESOURCE_GROUP \
  --cache-name MyHpcCache \
  --name MyStorageTarget \
  --storage-account $STORAGE_ACCOUNT_ID \
  --container-name $CONTAINER_NAME \
  --virtual-namespace-path "/myfilepath"
```

Set up client load balancing

Create an Azure Private DNS zone for the client-facing IP addresses using the az network private-dns zone create command. First define the environment variable PRIVATE_DNS_ZONE and specify a name for the zone.
```
PRIVATE_DNS_ZONE="myhpccache.local"
```
```
az network private-dns zone create \
  --resource-group $RESOURCE_GROUP \
  --name $PRIVATE_DNS_ZONE
```

Create a DNS link between the Azure Private DNS Zone and the VNet using the az network private-dns link vnet create command. Replace the value for --name.

az network private-dns link vnet create \
  --resource-group $RESOURCE_GROUP \
  --name MyDNSLink \
  --zone-name $PRIVATE_DNS_ZONE \
  --virtual-network $VNET_NAME \
  --registration-enabled true

Create the round-robin DNS name for the client-facing IP addresses using the az network private-dns record-set a create command. First, define the environment variables DNS_NAME, HPC_MOUNTS0, HPC_MOUNTS1, and HPC_MOUNTS2. Replace the value for the property DNS_NAME.

DNS_NAME="server"
HPC_MOUNTS0=$(az hpc-cache show --name "MyHpcCache" --resource-group $RESOURCE_GROUP --query "mountAddresses[0]" -o tsv | tr --delete '\r')
HPC_MOUNTS1=$(az hpc-cache show --name "MyHpcCache" --resource-group $RESOURCE_GROUP --query "mountAddresses[1]" -o tsv | tr --delete '\r')
HPC_MOUNTS2=$(az hpc-cache show --name "MyHpcCache" --resource-group $RESOURCE_GROUP --query "mountAddresses[2]" -o tsv | tr --delete '\r')

az network private-dns record-set a add-record -g $RESOURCE_GROUP -z $PRIVATE_DNS_ZONE -n $DNS_NAME -a $HPC_MOUNTS0

az network private-dns record-set a add-record -g $RESOURCE_GROUP -z $PRIVATE_DNS_ZONE -n $DNS_NAME -a $HPC_MOUNTS1

az network private-dns record-set a add-record -g $RESOURCE_GROUP -z $PRIVATE_DNS_ZONE -n $DNS_NAME -a $HPC_MOUNTS2

Create a persistent volume

Create a file named pv-nfs.yaml to define a persistent volume and then paste in the following manifest. Replace the values for the property server and path.

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs
spec:
  capacity:
    storage: 10000Gi
  accessModes:
    - ReadWriteMany
  mountOptions:
    - vers=3
  nfs:
    server: server.myhpccache.local
    path: /

Get the credentials for your Kubernetes cluster using the az aks get-credentials command.
```
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster
```
Create the persistent volume using the kubectl apply command.
```
kubectl apply -f pv-nfs.yaml
```
Verify the status of the persistent volume is Available using the kubectl describe command.
```
kubectl describe pv pv-nfs
```

Create the persistent volume claim

Create a file named pvc-nfs.yamlto define a persistent volume claim, and then paste the following manifest.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nfs
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ""
  resources:
    requests:
      storage: 100Gi

Create the persistent volume claim using the kubectl apply command.
```
kubectl apply -f pvc-nfs.yaml
```
Verify the status of the persistent volume claim is Bound using the kubectl describe command.
```
kubectl describe pvc pvc-nfs
```

Mount the HPC Cache with a pod

Create a file named nginx-nfs.yaml to define a pod that uses the persistent volume claim, and then paste the following manifest.

kind: Pod
apiVersion: v1
metadata:
  name: nginx-nfs
spec:
  containers:
 - image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    name: nginx-nfs
    command:
    - "/bin/sh"
    - "-c"
    - while true; do echo $(date) >> /mnt/azure/myfilepath/outfile; sleep 1; done
    volumeMounts:
    - name: disk01
      mountPath: /mnt/azure
  volumes:
 - name: disk01
    persistentVolumeClaim:
      claimName: pvc-nfs

Create the pod using the kubectl apply command.
```
kubectl apply -f nginx-nfs.yaml
```
Verify the pod is running using the kubectl describe command.
```
kubectl describe pod nginx-nfs
```
Verify your volume is mounted in the pod using the kubectl exec command to connect to the pod.
```
kubectl exec -it nginx-nfs -- sh
```
To check if the volume is mounted, run df in its human-readable format using the --human-readable (-h for short) option.
```
df -h
```
The following example resembles output returned from the command:
```
Filesystem             Size  Used Avail Use% Mounted on
...
server.myhpccache.local:/myfilepath 8.0E         0      8.0E   0% /mnt/azure/myfilepath
...
```

Next steps

For more information on Azure HPC Cache, see HPC Cache overview.
For more information on using NFS with AKS, see Manually create and use a Network File System (NFS) Linux Server volume with AKS.

Share via

Integrate Azure HPC Cache with Azure Kubernetes Service (AKS)

Before you begin

Install the `hpc-cache` Azure CLI extension

Register the StorageCache feature flag

Create the Azure HPC Cache

Create and configure Azure storage

Set up client load balancing

Create a persistent volume

Create the persistent volume claim

Mount the HPC Cache with a pod

Next steps

Feedback

Additional resources

Share via

Integrate Azure HPC Cache with Azure Kubernetes Service (AKS)

Before you begin

Install the hpc-cache Azure CLI extension

Register the StorageCache feature flag

Create the Azure HPC Cache

Create and configure Azure storage

Set up client load balancing

Create a persistent volume

Create the persistent volume claim

Mount the HPC Cache with a pod

Next steps

Feedback

Additional resources

Install the `hpc-cache` Azure CLI extension