We have an AKS cluster set up with an app gateway. We need both autoscaling (only available in v2) and the ability to route traffic to/from the Internet so that Internet<>App Gateway always hops across the the nat gateway.
In other words, we want the traffic to route:
Ingress: nat gateway public IP > nat gateway > app gateway
Egress: app gateway > nat gateway > nat gateway public IP
And we want to prohibit:
Ingress: Internet > app gateway public IP > app gateway
Egress: app gateway > app gateway public IP > Internet
Since v2 requires a public IP and open ports, we can't simply block all inbound on its public IP and force use of the nat gateway IP as that is not allowed by Azure. I'm thinking this needs to be done with UDRs, but I can't figure how to do it in a way Azure finds acceptable.
Is there a solution for this?
Environment: The nat gateway and app gateway are located in the hub vnet. Each have their own subnet. The AKS cluster is located in a spoke vnet with its own subnet. The two vnets are peered together. Currently the nat gateway and app gateway both have their own public IPs.