Povratne informacije Uredi

Customize cluster egress with a User-Defined Route

  • Članak
  • 3 min. za čitanje

Egress from an AKS cluster can be customized to fit specific scenarios. By default, AKS will provision a Standard SKU Load Balancer to be set up and used for egress. However, the default setup may not meet the requirements of all scenarios if public IPs are disallowed or additional hops are required for egress.

This article walks through how to customize a cluster's egress route to support custom network scenarios, such as those which disallows public IPs and requires the cluster to sit behind a network virtual appliance (NVA).

Prerequisites

  • Azure CLI version 2.0.81 or greater
  • API version of 2020-01-01 or greater

Limitations

  • OutboundType can only be defined at cluster create time and can't be updated afterwards.
  • Setting outboundType requires AKS clusters with a vm-set-type of VirtualMachineScaleSets and load-balancer-sku of Standard.
  • Setting outboundType to a value of UDR requires a user-defined route with valid outbound connectivity for the cluster.
  • Setting outboundType to a value of UDR implies the ingress source IP routed to the load-balancer may not match the cluster's outgoing egress destination address.

Overview of outbound types in AKS

An AKS cluster can be customized with a unique outboundType of type loadBalancer or userDefinedRouting.

Important

Outbound type impacts only the egress traffic of your cluster. For more information, see setting up ingress controllers.

Note

You can use your own route table with UDR and kubenet networking. Make sure you cluster identity (service principal or managed identity) has Contributor permissions to the custom route table.

Outbound type of loadBalancer

If loadBalancer is set, AKS completes the following configuration automatically. The load balancer is used for egress through an AKS assigned public IP. An outbound type of loadBalancer supports Kubernetes services of type loadBalancer, which expect egress out of the load balancer created by the AKS resource provider.

The following configuration is done by AKS.

  • A public IP address is provisioned for cluster egress.
  • The public IP address is assigned to the load balancer resource.
  • Backend pools for the load balancer are set up for agent nodes in the cluster.

Below is a network topology deployed in AKS clusters by default, which use an outboundType of loadBalancer.

Diagram shows ingress I P and egress I P, where the ingress I P directs traffic to a load balancer, which directs traffic to and from an internal cluster and other traffic to the egress I P, which directs traffic to the Internet, M C R, Azure required services, and the A K S Control Plane.

Outbound type of userDefinedRouting

Note

Using outbound type is an advanced networking scenario and requires proper network configuration.

If userDefinedRouting is set, AKS won't automatically configure egress paths. The egress setup must be done by you.

The AKS cluster must be deployed into an existing virtual network with a subnet that has been previously configured because when not using standard load balancer (SLB) architecture, you must establish explicit egress. As such, this architecture requires explicitly sending egress traffic to an appliance like a firewall, gateway, proxy or to allow the Network Address Translation (NAT) to be done by a public IP assigned to the standard load balancer or appliance.

Load balancer creation with userDefinedRouting

AKS clusters with an outbound type of UDR receive a standard load balancer (SLB) only when the first Kubernetes service of type 'loadBalancer' is deployed. The load balancer is configured with a public IP address for inbound requests and a backend pool for inbound requests. Inbound rules are configured by the Azure cloud provider, but no outbound public IP address or outbound rules are configured as a result of having an outbound type of UDR. Your UDR will still be the only source for egress traffic.

Azure load balancers don't incur a charge until a rule is placed.

Deploy a cluster with outbound type of UDR and Azure Firewall

To illustrate the application of a cluster with outbound type using a user-defined route, a cluster can be configured on a virtual network with an Azure Firewall on its own subnet. See this example on the restrict egress traffic with Azure firewall example.

Important

Outbound type of UDR requires there is a route for 0.0.0.0/0 and next hop destination of NVA (Network Virtual Appliance) in the route table. The route table already has a default 0.0.0.0/0 to Internet, without a Public IP to SNAT just adding this route will not provide you egress. AKS will validate that you don't create a 0.0.0.0/0 route pointing to the Internet but instead to NVA or gateway, etc. When using an outbound type of UDR, a load balancer public IP address for inbound requests is not created unless a service of type loadbalancer is configured. A public IP address for outbound requests is never created by AKS if an outbound type of UDR is set.

Next steps

See Azure networking UDR overview.

See how to create, change, or delete a route table.