Use Calico network policy to secure pod traffic

Article
02/16/2024

Applies to: AKS on Azure Stack HCI 22H2, AKS on Windows Server

Use this step-by-step guide to verify and try out basic pod-to-pod connectivity and to use Calico network policies in a cluster in an AKS enabled by Azure Arc deployment. This article describes how to create client and server pods on Linux and Windows nodes, verify connectivity between the pods, and then apply a basic network policy to isolate pod traffic in AKS Arc.

Prerequisites

To deploy AKS Arc, follow the steps to set up an AKS host.

To use this guide, you need:

An AKS workload cluster.
At least one Windows worker node deployed in the cluster.
At least one Linux worker node deployed in the cluster.
The Calico network plug-in must be enabled when creating the workload cluster. If this plug-in wasn't enabled, see New-AksHciCluster for instructions.

Create pods on Linux nodes

First, create a client pod, busybox, and server pod, nginx, on the Linux nodes.

Create a YAML file called policy-demo-linux.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: calico-demo

---

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: busybox
  name: busybox
  namespace: calico-demo
spec:
  containers:
  - args:
    - /bin/sh
    - -c
    - sleep 360000
    image: busybox:1.28
    imagePullPolicy: Always
    name: busybox
  nodeSelector:
    beta.kubernetes.io/os: linux

---

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: calico-demo
spec:
  containers:
  - name: nginx
    image: nginx:1.8
    ports:
    - containerPort: 80
  nodeSelector:
    beta.kubernetes.io/os: linux

Apply the policy-demo-linux.yaml file to the Kubernetes cluster

Open a PowerShell window, and load the credentials for your target cluster using the Get-AksHciCredential command.

Next, use kubectl to apply the policy-demo-linux.yaml configuration, as follows:

kubectl apply -f policy-demo-linux.yaml

Create pods on Window nodes

Create a client pod pwsh and server pod porter on the Windows nodes.

Note

The pods' manifest uses images based on mcr.microsoft.com/windows/servercore:1809. If you are using a more recent Windows Server version, update the manifests to use a Server Core image that matches your Windows Server version.

Create the policy-demo-windows.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pwsh
  namespace: calico-demo
  labels:
    app: pwsh
spec:
  containers:
  - name: pwsh
    image: mcr.microsoft.com/windows/servercore:1809
    args:
    - powershell.exe
    - -Command
    - "Start-Sleep 360000"
    imagePullPolicy: IfNotPresent
  nodeSelector:
    kubernetes.io/os: windows
---
apiVersion: v1
kind: Pod
metadata:
  name: porter
  namespace: calico-demo
  labels:
    app: porter
spec:
  containers:
  - name: porter
    image: calico/porter:1809
    ports:
    - containerPort: 80
    env:
    - name: SERVE_PORT_80
      value: This is a Calico for Windows demo.
    imagePullPolicy: IfNotPresent
  nodeSelector:
    kubernetes.io/os: windows

Apply the policy-demo-windows.yaml file to the Kubernetes cluster

Open a PowerShell window, and load the credentials for your target cluster using the Get-AksHciCredential command.

Next, use kubectl to apply the policy-demo-windows.yaml configuration:

kubectl apply -f policy-demo-windows.yaml

Verify the four pods are created and running

Note

Depending on your network download speed, it may take time to launch the Windows pods.

Open a PowerShell window, and load the credentials for your target cluster using the Get-AksHciCredential command.

Next, use kubectl to list the pods in the calico-demo namespace:

kubectl get pods --namespace calico-demo

You should see output similar to the following example:

NAME      READY   STATUS              RESTARTS   AGE
busybox   1/1     Running             0          4m14s
nginx     1/1     Running             0          4m14s
porter    0/1     ContainerCreating   0          74s
pwsh      0/1     ContainerCreating   0          2m9s

Repeat the command every few minutes until the output shows all four pods in the Running state.

NAME      READY   STATUS    RESTARTS   AGE
busybox   1/1     Running   0          7m24s
nginx     1/1     Running   0          7m24s
porter    1/1     Running   0          4m24s
pwsh      1/1     Running   0          5m19s

Check connectivity between pods on Linux and Windows nodes

Now that the client and server pods are running on both Linux and Windows nodes, verify that client pods on Linux nodes can reach server pods on Windows nodes.

Open a PowerShell window, and load the credentials for your target cluster using the Get-AksHciCredential command.

Use kubectl to determine the porter pod IP address:

kubectl get pod porter --namespace calico-demo -o 'jsonpath={.status.podIP}'

Sign in to the busybox pod and try to reach the porter pod on port 80. Replace the '<porter_ip>' tag with the IP address returned from the previous command.
```
kubectl exec --namespace calico-demo busybox -- nc -vz <porter_ip> 80
```
You can also combine both of the above steps:
```
kubectl exec --namespace calico-demo busybox -- nc -vz $(kubectl get pod porter --namespace calico-demo -o 'jsonpath={.status.podIP}') 80
```
If the connection from the busybox pod to the porter pod succeeds, you get output similar to the following example:
```
192.168.40.166 (192.168.40.166:80) open
```
Note

The IP addresses returned can vary depending on your environment setup.

Verify that the pwsh pod can reach the nginx pod:

kubectl exec --namespace calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po nginx -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5

If the connection succeeds, you see output similar to:

StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html>
                    <head>
                    <title>Welcome to nginx!</title>
                    <style>
                        body {
                            width: 35em;
                            margin: 0 auto;
                            font-family: Tahoma, Verdana, Arial, sans-serif;
                        }
                    </style>
                    <...

Verify that the pwsh pod can reach the porter pod:

kubectl exec --namespace calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5

If that succeeds, you see output similar to the following example:

StatusCode        : 200
StatusDescription : OK
Content           : This is a Calico for Windows demo.
RawContent        : HTTP/1.1 200 OK
                    Content-Length: 49
                    Content-Type: text/plain; charset=utf-8
                    Date: Fri, 21 Aug 2020 22:45:46 GMT

                    This is a Calico for Windows demo.
Forms             :
Headers           : {[Content-Length, 49], [Content-Type, text/plain;
                    charset=utf-8], [Date, Fri, 21 Aug 2020 22:45:46 GMT]}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        :
RawContentLength  : 49

You have now verified that communication is possible between all pods in the application.

Apply the policy to the Windows client pod

In a real-world deployment, you want to make sure only pods that are supposed to communicate with each other are allowed to do so. To achieve this, you apply a basic network policy that allows only the busybox pod to reach the porter pod.

Create the network-policy.yaml file

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-busybox
  namespace: calico-demo
spec:
  podSelector:
    matchLabels:
      app: porter
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: busybox
    ports:
    - protocol: TCP
      port: 80

Apply the network-policy.yaml file

Open a PowerShell window.
Load the credentials for your target cluster using the Get-AksHciCredential command.
Use kubectl to apply the network-policy.yaml file:
```
kubectl apply -f network-policy.yaml
```

Verify the policy is in effect

With the policy in place, the busybox pod should still be able to reach the porter pod. As noted previously, you can combine the steps in the command line:

kubectl exec --namespace calico-demo busybox -- nc -vz $(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') 80

However, the pwsh pod won't be able to reach the porter pod:

kubectl exec --namespace calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5

The request times out with a message like this one:

Invoke-WebRequest : The operation has timed out.
At line:1 char:1
+ Invoke-WebRequest -Uri http://192.168.40.166 -UseBasicParsing -Timeout ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:Htt
pWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
command terminated with exit code 1

In this demo, we configured pods on Linux and Windows nodes, verified basic pod connectivity, and tried a basic network policy to isolate pod-to-pod traffic.

As the final step, you can clean up all of the demo resources:

kubectl delete namespace calico-demo

Next steps

In this article, you learned how to secure traffic between pods using network policies. Next, you can:

Use Calico network policy to secure pod traffic

Prerequisites

Create pods on Linux nodes

Create a YAML file called policy-demo-linux.yaml

Apply the policy-demo-linux.yaml file to the Kubernetes cluster

Create pods on Window nodes

Create the policy-demo-windows.yaml

Apply the policy-demo-windows.yaml file to the Kubernetes cluster

Verify the four pods are created and running

Check connectivity between pods on Linux and Windows nodes

Apply the policy to the Windows client pod

Create the network-policy.yaml file

Apply the network-policy.yaml file

Verify the policy is in effect

Next steps

Feedback

Feedback

Additional resources