Adapt apps for mixed-OS Kubernetes clusters using node selectors or taints and tolerations

Applies to: AKS on Azure Stack HCI, AKS runtime on Windows Server 2019 Datacenter

Azure Kubernetes Service on Azure Stack HCI enables you to run Kubernetes clusters with both Linux and Windows nodes, but requires you to make small edits to your apps for use in these mixed-OS clusters. In this how-to guide, you learn how to ensure your application gets scheduled on the right host OS using either node selectors or taints and tolerations.

This how-to guide assumes a basic understanding of Kubernetes concepts. For more information, see Kubernetes core concepts for Azure Kubernetes Service on Azure Stack HCI.

Node Selector

Node Selector is a simple field in the pod specification YAML that constrains pods to only be scheduled onto healthy nodes matching the operating system. In your pod specification YAML, specify a nodeSelector - Windows or Linux, as shown in the examples below.

kubernetes.io/os = Windows

or,

kubernetes.io/os = Linux

For more information on nodeSelectors, visit node selectors.

Taints and tolerations

Taints and tolerations work together to ensure that pods aren't scheduled on nodes unintentionally. A node can be "tainted" to not accept pods that don't explicitly tolerate its taint through a "toleration" in the pod specification YAML.

Windows OS nodes in Azure Kubernetes Service on Azure Stack HCI can be tainted with the following key-value pair. Users shouldn't use a different one.

node.kubernetes.io/os=Windows:NoSchedule

Run kubectl get and identify the Windows worker nodes you want to taint.

kubectl get nodes --all-namespaces -o=custom-columns=NAME:.metadata.name,OS:.status.nodeInfo.operatingSystem

Output:

NAME                                     OS
my-aks-hci-cluster-control-plane-krx7j   linux
my-aks-hci-cluster-md-md-1-5h4bl         windows
my-aks-hci-cluster-md-md-1-5xlwz         windows

Taint Windows server worker nodes using kubectl taint node.

kubectl taint node my-aks-hci-cluster-md-md-1-5h4bl node.kubernetes.io/os=Windows:NoSchedule
kubectl taint node my-aks-hci-cluster-md-md-1-5xlwz node.kubernetes.io/os=Windows:NoSchedule

You specify a toleration for a pod in the pod specification YAML. The following toleration "matches" the taint created by the kubectl taint line above, and thus a pod with the toleration would be able to schedule onto my-aks-hci-cluster-md-md-1-5h4bl or my-aks-hci-cluster-md-md-1-5xlwz:

tolerations:
- key: node.kubernetes.io/os
  operator: Equal
  value: Windows
  effect: NoSchedule

For more information on taints and tolerations, visit Taints and Tolerations.

Next steps

In this how-to guide, you learned how to add node selectors or taints and tolerations to your Kubernetes clusters using kubectl. Next, you can: