在 Azure Kubernetes Service (AKS) 中設定 Azure CNI 網路Configure Azure CNI networking in Azure Kubernetes Service (AKS)

根據預設, AKS 叢集會使用kubenet, 並為您建立虛擬網路和子網。By default, AKS clusters use kubenet, and a virtual network and subnet are created for you. 使用 kubenet,節點會從虛擬網路子網路取得 IP 位址。With kubenet, nodes get an IP address from a virtual network subnet. 接著會在節點上設定網路位址轉譯 (NAT),而 Pod 會接收「隱藏」於節點 IP 後面的 IP 位址。Network address translation (NAT) is then configured on the nodes, and pods receive an IP address "hidden" behind the node IP. 此方法可減少您需要在網路空間中保留,以供 Pod 使用的 IP 位址數目。This approach reduces the number of IP addresses that you need to reserve in your network space for pods to use.

使用Azure 容器網路介面 (CNI), 每個 pod 都會從子網取得 IP 位址, 而且可以直接存取。With Azure Container Networking Interface (CNI), every pod gets an IP address from the subnet and can be accessed directly. 這些 IP 位址在您的網路空間中必須是唯一的,且必須事先規劃。These IP addresses must be unique across your network space, and must be planned in advance. 每個節點都有一個組態參數,用於所支援的最大 Pod 數目。Each node has a configuration parameter for the maximum number of pods that it supports. 然後,為該節點預先保留每個節點的相同 IP 位址數目。The equivalent number of IP addresses per node are then reserved up front for that node. 此方法需要更多規劃,並且通常會導致 IP 位址耗盡,或者隨著應用程式需求增加,需要在更大型子網路中重建叢集。This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow.

本文示範如何使用 Azure CNI 網路,針對 AKS 叢集建立和使用虛擬網路子網路。This article shows you how to use Azure CNI networking to create and use a virtual network subnet for an AKS cluster. 如需網路選項和考慮的詳細資訊, 請參閱Kubernetes 和 AKS 的網路概念For more information on network options and considerations, see Network concepts for Kubernetes and AKS.

必要條件Prerequisites

  • 適用於 AKS 叢集的虛擬網路必須允許輸出網際網路連線.The virtual network for the AKS cluster must allow outbound internet connectivity.
  • 請勿在相同子網路中建立多個 AKS 叢集。Don't create more than one AKS cluster in the same subnet.
  • AKS 叢集可能不會169.254.0.0/16使用172.30.0.0/16 172.31.0.0/16、、或192.0.2.0/24作為 Kubernetes 服務位址範圍。AKS clusters may not use 169.254.0.0/16, 172.30.0.0/16, 172.31.0.0/16, or 192.0.2.0/24 for the Kubernetes service address range.
  • AKS 叢集所使用的服務主體在您虛擬網路內的子網路上必須至少具有網路參與者權限。The service principal used by the AKS cluster must have at least Network Contributor permissions on the subnet within your virtual network. 如果您想要定義自訂角色,而不使用內建的網路參與者角色,則需要下列權限:If you wish to define a custom role instead of using the built-in Network Contributor role, the following permissions are required:
    • Microsoft.Network/virtualNetworks/subnets/join/action
    • Microsoft.Network/virtualNetworks/subnets/read

規劃叢集的 IP 位址Plan IP addressing for your cluster

使用 Azure CNI 網路設定的叢集需要進行額外的規劃。Clusters configured with Azure CNI networking require additional planning. 您的虛擬網路及其子網路的大小必須配合您規劃要執行的 Pod 數目以及叢集的節點數目。The size of your virtual network and its subnet must accommodate the number of pods you plan to run and the number of nodes for the cluster.

Pod 和叢集節點的 IP 位址會從虛擬網路內的指定子網路來指派。IP addresses for the pods and the cluster's nodes are assigned from the specified subnet within the virtual network. 每個節點都以主要 IP 位址進行設定。Each node is configured with a primary IP address. 根據預設,Azure CNI 預先設定的 30 個額外 IP 位址會指派給在節點上排程的 Pod。By default, 30 additional IP addresses are pre-configured by Azure CNI that are assigned to pods scheduled on the node. 當您將叢集相應放大時,每個節點同樣也會設定子網路中的 IP 位址。When you scale out your cluster, each node is similarly configured with IP addresses from the subnet. 您也可以檢視每個節點的最大 Pod 數目You can also view the maximum pods per node.

重要

所需的 IP 位址數目應包含針對升級和調整作業考量的數目。The number of IP addresses required should include considerations for upgrade and scaling operations. 如果設定只支援固定節點數目的 IP 位址範圍,將無法升級或擴展您的叢集。If you set the IP address range to only support a fixed number of nodes, you cannot upgrade or scale your cluster.

  • 當您升級 AKS 叢集時,會部署新的節點到叢集中。When you upgrade your AKS cluster, a new node is deployed into the cluster. 服務和工作負載會開始在新的節點上執行,並移除叢集中較舊的節點。Services and workloads begin to run on the new node, and an older node is removed from the cluster. 此輪流升級程序需要至少一個額外的 IP 位址區塊以供使用。This rolling upgrade process requires a minimum of one additional block of IP addresses to be available. 因此您的節點計數為 n + 1Your node count is then n + 1.

    • 當您使用 Windows Server 節點集區時 (目前在 AKS 中處於預覽狀態), 這項考慮特別重要。This consideration is particularly important when you use Windows Server node pools (currently in preview in AKS). AKS 中的 windows Server 節點不會自動套用 Windows 更新, 而是改為在節點集區上執行升級。Windows Server nodes in AKS do not automatically apply Windows Updates, instead you perform an upgrade on the node pool. 此升級會使用最新的 Window Server 2019 基底節點映射和安全性修補程式來部署新的節點。This upgrade deploys new nodes with the latest Window Server 2019 base node image and security patches. 如需有關升級 Windows Server 節點集區的詳細資訊, 請參閱升級 AKS 中的節點集區。For more information on upgrading a Windows Server node pool, see Upgrade a node pool in AKS.
  • 當您擴展 AKS 叢集時,會部署新的節點到叢集中。When you scale an AKS cluster, a new node is deployed into the cluster. 服務和工作負載會開始在新的節點上執行。Services and workloads begin to run on the new node. 您的 IP 位址範圍必須將您可能想要相應增加的節點數和您的叢集可支援的 Pod 數目納入考量。Your IP address range needs to take into considerations how you may want to scale up the number of nodes and pods your cluster can support. 也應包含一個額外的節點以用於升級作業。One additional node for upgrade operations should also be included. 因此您的節點計數為 n + number-of-additional-scaled-nodes-you-anticipate + 1Your node count is then n + number-of-additional-scaled-nodes-you-anticipate + 1.

如果預期您的節點會執行最大數目的 Pod,並定期終結及部署 Pod 時,也應該考慮為每個節點提供一些額外的 IP 位址。If you expect your nodes to run the maximum number of pods, and regularly destroy and deploy pods, you should also factor in some additional IP addresses per node. 這些納入考量的額外 IP 位址列可能需要幾秒鐘的時間刪除服務,並且為部署新服務而釋放 IP 地址並取得位址。These additional IP addresses take into consideration it may take a few seconds for a service to be deleted and the IP address released for a new service to be deployed and acquire the address.

適用於 AKS 叢集的 IP 位址方案會由一個虛擬網路、至少一個適用於節點和 Pod 的子網路,以及一個 Kubernetes 服務位址範圍所組成。The IP address plan for an AKS cluster consists of a virtual network, at least one subnet for nodes and pods, and a Kubernetes service address range.

位址範圍 / Azure 資源Address range / Azure resource 限制和調整大小Limits and sizing
虛擬網路Virtual network Azure 虛擬網路可以和 /8 一樣大,但可能只有 65,536 個已設定的 IP 位址。The Azure virtual network can be as large as /8, but is limited to 65,536 configured IP addresses.
SubnetSubnet 必須大到足以容納節點、Pod,以及可能會在您叢集中佈建的所有 Kubernetes 和 Azure 資源。Must be large enough to accommodate the nodes, pods, and all Kubernetes and Azure resources that might be provisioned in your cluster. 例如,如果您部署內部 Azure Load Balancer,其前端 IP 會從叢集子網路配置,而不是從公用 IP 配置。For example, if you deploy an internal Azure Load Balancer, its front-end IPs are allocated from the cluster subnet, not public IPs. 子網路大小也應該考量帳戶升級作業或未來的擴展需求。The subnet size should also take into account upgrade operations or future scaling needs.

若要計算包括用於升級作業之額外節點的最小子網路大小:(number of nodes + 1) + ((number of nodes + 1) * maximum pods per node that you configure)To calculate the minimum subnet size including an additional node for upgrade operations: (number of nodes + 1) + ((number of nodes + 1) * maximum pods per node that you configure)

50 個節點叢集的範例:(51) + (51 * 30 (default)) = 1,581 (/21 或更大)Example for a 50 node cluster: (51) + (51 * 30 (default)) = 1,581 (/21 or larger)

擁有 50 個節點的叢集範例,其中也包含相應增加額外 10 個節點的佈建:(61) + (61 * 30 (default)) = 1,891 (/21 或更大)Example for a 50 node cluster that also includes provision to scale up an additional 10 nodes: (61) + (61 * 30 (default)) = 1,891 (/21 or larger)

如果您未指定每個節點的最大 Pod 數目,當您建立叢集時,每個節點的最大 Pod 數目設定為 30。If you don't specify a maximum number of pods per node when you create your cluster, the maximum number of pods per node is set to 30. IP 位址所需的最小數目是根據該值。The minimum number of IP addresses required is based on that value. 如果您以不同的最大值來計算最小 IP 位址需求,請參閱如何設定每個節點的最大 Pod 數目,在您部署叢集時設定此值。If you calculate your minimum IP address requirements on a different maximum value, see how to configure the maximum number of pods per node to set this value when you deploy your cluster.

Kubernetes 服務位址範圍Kubernetes service address range 此範圍不應由此虛擬網路上或連線到此虛擬網路的任何網路元素所使用。This range should not be used by any network element on or connected to this virtual network. 服務位址 CIDR 必須小於 /12。Service address CIDR must be smaller than /12.
Kubernetes DNS 服務 IP 位址Kubernetes DNS service IP address 將由叢集服務探索 (kube-dns) 所使用之 Kubernetes 服務位址範圍內的 IP 位址。IP address within the Kubernetes service address range that will be used by cluster service discovery (kube-dns). 請勿使用您位址範圍中的第一個 IP 位址,例如 .1。Don't use the first IP address in your address range, such as .1. 您子網路範圍內的第一個位址會用於 kubernetes.default.svc.cluster.local 位址。The first address in your subnet range is used for the kubernetes.default.svc.cluster.local address.
Docker 橋接器位址Docker bridge address 用來作為節點上 Docker 橋接器 IP 位址的 IP 位址 (採用 CIDR 標記法)。IP address (in CIDR notation) used as the Docker bridge IP address on nodes. 此 CIDR 會與節點上的容器數目相關聯。This CIDR is tied to the number of containers on the node. 預設值為 172.17.0.1/16。Default of 172.17.0.1/16.

每個節點的最大 Pod 數目Maximum pods per node

AKS 叢集中每個節點的 pod 數目上限為250。The maximum number of pods per node in an AKS cluster is 250. 每個節點「預設」的最大 Pod 數目,會根據 kubenetAzure CNI 網路以及叢集部署的方法而有所不同。The default maximum number of pods per node varies between kubenet and Azure CNI networking, and the method of cluster deployment.

部署方法Deployment method Kubenet 預設值Kubenet default Azure CNI 預設值Azure CNI default 可在部署時設定Configurable at deployment
Azure CLIAzure CLI 110110 3030 是 (最多 250)Yes (up to 250)
Resource Manager 範本Resource Manager template 110110 3030 是 (最多 250)Yes (up to 250)
入口網站Portal 110110 3030 No

設定最大值 - 新叢集Configure maximum - new clusters

只能在叢集部署階段設定每一節點的 Pod 數目上限。You're able to configure the maximum number of pods per node only at cluster deployment time. 如果您使用 Azure CLI 或 Resource Manager 範本進行部署, 則可以將每個節點的最大 pod 值設定為高達250。If you deploy with the Azure CLI or with a Resource Manager template, you can set the maximum pods per node value as high as 250.

網路功能Networking 最小值Minimum 最大值Maximum
Azure CNIAzure CNI 3030 250250
KubenetKubenet 3030 110110

注意

AKS 服務會嚴格強制執行上表中的最小值。The minimum value in the table above is strictly enforced by the AKS service. 您不能將 maxPods 值設定為低於所顯示的最小值, 因為這麼做可能會導致叢集無法啟動。You can not set a maxPods value lower than the minimum shown as doing so can prevent the cluster from starting.

  • Azure CLI:當您--max-pods使用az aks create命令部署叢集時, 請指定引數。Azure CLI: Specify the --max-pods argument when you deploy a cluster with the az aks create command. 最大值為250。The maximum value is 250.
  • Resource Manager 範本:當您使用 Resource Manager 範本部署叢集時,請指定 ManagedClusterAgentPoolProfile 物件中的 maxPods 屬性。Resource Manager template: Specify the maxPods property in the ManagedClusterAgentPoolProfile object when you deploy a cluster with a Resource Manager template. 最大值為250。The maximum value is 250.
  • Azure 入口網站︰當您使用 Azure 入口網站部署叢集時,您無法變更每一節點的 Pod 數目上限。Azure portal: You can't change the maximum number of pods per node when you deploy a cluster with the Azure portal. 使用 Azure 入口網站進行部署時,Azure CNI 網路叢集限制為每一節點 30 個 Pod。Azure CNI networking clusters are limited to 30 pods per node when you deploy using the Azure portal.

設定最大值 - 現有叢集Configure maximum - existing clusters

您無法在現有 AKS 叢集上變更每個節點的 Pod 數目上限。You can't change the maximum pods per node on an existing AKS cluster. 只有在您一開始部署叢集時,才能調整此數目。You can adjust the number only when you initially deploy the cluster.

部署參數Deployment parameters

當您建立 AKS 叢集時,可針對 Azure CNI 網路設定下列參數:When you create an AKS cluster, the following parameters are configurable for Azure CNI networking:

虛擬網路:要作為 Kubernetes 叢集部署目的地的虛擬網路。Virtual network: The virtual network into which you want to deploy the Kubernetes cluster. 如果您要為叢集建立新的虛擬網路,請選取 [新建] 並遵循<建立虛擬網路>一節中的步驟。If you want to create a new virtual network for your cluster, select Create new and follow the steps in the Create virtual network section. 如需有關 Azure 虛擬網路限制和配額的資訊,請參閱 Azure 訂用帳戶和服務限制、配額及條件約束For information about the limits and quotas for an Azure virtual network, see Azure subscription and service limits, quotas, and constraints.

子網路:虛擬網路內要用來部署叢集的子網路。Subnet: The subnet within the virtual network where you want to deploy the cluster. 如果您要在虛擬網路中為叢集建立新的子網路,請選取 [新建] 並遵循<建立子網路>一節中的步驟。If you want to create a new subnet in the virtual network for your cluster, select Create new and follow the steps in the Create subnet section. 混合式連線的位址範圍不應該與您環境中的任何其他虛擬網路重疊。For hybrid connectivity, the address range shouldn't overlap with any other virtual networks in your environment.

Kubernetes 服務位址範圍:這是 Kubernetes 指派給您叢集中內部服務的一組虛擬 ip。Kubernetes service address range: This is the set of virtual IPs that Kubernetes assigns to internal services in your cluster. 您可以使用任何符合下列需求的私人位址範圍:You can use any private address range that satisfies the following requirements:

  • 不得在叢集的虛擬網路 IP 位址範圍內Must not be within the virtual network IP address range of your cluster
  • 不得與叢集虛擬網路對等的任何其他虛擬網路重疊Must not overlap with any other virtual networks with which the cluster virtual network peers
  • 不得與任何內部部署 IP 重疊Must not overlap with any on-premises IPs
  • 不得在169.254.0.0/16 172.30.0.0/16、 、或範圍內172.31.0.0/16``192.0.2.0/24Must not be within the ranges 169.254.0.0/16, 172.30.0.0/16, 172.31.0.0/16, or 192.0.2.0/24

雖然技術上有可能指定與您叢集相同虛擬網路內的服務位址範圍,但不建議這麼做。Although it's technically possible to specify a service address range within the same virtual network as your cluster, doing so is not recommended. 如果使用重疊的 IP 範圍,就會造成無法預期的行為。Unpredictable behavior can result if overlapping IP ranges are used. 如需詳細資訊,請參閱本文的常見問題集一節。For more information, see the FAQ section of this article. 如需 Kubernetes 服務的詳細資訊, 請參閱 Kubernetes 檔中的服務For more information on Kubernetes services, see Services in the Kubernetes documentation.

Kubernetes DNS 服務 IP 位址:叢集 DNS 服務的 IP 位址。Kubernetes DNS service IP address: The IP address for the cluster's DNS service. 此位址必須位於 Kubernetes 服務位址範圍內。This address must be within the Kubernetes service address range. 請勿使用您位址範圍中的第一個 IP 位址,例如 .1。Don't use the first IP address in your address range, such as .1. 您子網路範圍內的第一個位址會用於 kubernetes.default.svc.cluster.local 位址。The first address in your subnet range is used for the kubernetes.default.svc.cluster.local address.

Docker 橋接器位址:Docker 橋接器網路位址代表所有 Docker 安裝中出現的預設docker0橋接器網路位址。Docker Bridge address: The Docker bridge network address represents the default docker0 bridge network address present in all Docker installations. 雖然 AKS 叢集或 pod 本身不會使用docker0橋接器,但您必須將此位址設定為繼續支援在 AKS 叢集中的docker build等案例。While docker0 bridge is not used by AKS clusters or the pods themselves, you must set this address to continue to support scenarios such as docker build within the AKS cluster. 您必須為 Docker 橋接器網路位址選取 CIDR,否則 Docker 會自動挑選可能與其他 CIDRs 發生衝突的子網。It is required to select a CIDR for the Docker bridge network address because otherwise Docker will pick a subnet automatically which could conflict with other CIDRs. 您必須挑選不會與網路上的其餘 CIDRs 發生衝突的位址空間,包括叢集的服務 CIDR 和 pod CIDR。You must pick an address space that does not collide with the rest of the CIDRs on your networks, including the cluster's service CIDR and pod CIDR.

設定網路功能 - CLIConfigure networking - CLI

當您使用 Azure CLI 來建立 AKS 叢集時,也可以設定 Azure CLI 網路。When you create an AKS cluster with the Azure CLI, you can also configure Azure CNI networking. 使用下列命令來建立新的 AKS 叢集,並啟用 Azure CNI 網路。Use the following commands to create a new AKS cluster with Azure CNI networking enabled.

首先,針對將聯結 AKS 叢集的現有子網路取得子網路資源識別碼:First, get the subnet resource ID for the existing subnet into which the AKS cluster will be joined:

$ az network vnet subnet list \
    --resource-group myVnet \
    --vnet-name myVnet \
    --query "[0].id" --output tsv

/subscriptions/<guid>/resourceGroups/myVnet/providers/Microsoft.Network/virtualNetworks/myVnet/subnets/default

使用az aks create命令--network-plugin azure搭配引數, 以建立具有 advanced 網路功能的叢集。Use the az aks create command with the --network-plugin azure argument to create a cluster with advanced networking. 使用在上一個步驟中收集的子網路識別碼來更新 --vnet-subnet-id 值:Update the --vnet-subnet-id value with the subnet ID collected in the previous step:

az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --network-plugin azure \
    --vnet-subnet-id <subnet-id> \
    --docker-bridge-address 172.17.0.1/16 \
    --dns-service-ip 10.2.0.10 \
    --service-cidr 10.2.0.0/24 \
    --generate-ssh-keys

設定網路功能 - 入口網站Configure networking - portal

下列擷取自 Azure 入口網站的螢幕擷取畫面,會顯示 AKS 叢集建立期間對這些設定進行配置的範例:The following screenshot from the Azure portal shows an example of configuring these settings during AKS cluster creation:

Azure 入口網站中的進階網路組態

常見問題集Frequently asked questions

下列問題和解答適用於 Azure CNI 網路設定。The following questions and answers apply to the Azure CNI networking configuration.

  • 是否可以在叢集子網路中部署 VM?Can I deploy VMs in my cluster subnet?

    資料分割No. 不支援在 Kubernetes 叢集所使用的子網路中部署 VM。Deploying VMs in the subnet used by your Kubernetes cluster is not supported. 虛擬機器可部署在相同虛擬網路中,但不能部署在不同的子網路。VMs may be deployed in the same virtual network, but in a different subnet.

  • 是否可以針對個別 Pod 設定網路原則?Can I configure per-pod network policies?

    是, AKS 中有提供 Kubernetes 網路原則。Yes, Kubernetes network policy is available in AKS. 若要開始, 請參閱在 AKS 中使用網路原則來保護 pod 之間的流量To get started, see Secure traffic between pods by using network policies in AKS.

  • 是否可以設定可部署到節點的 Pod 數目上限?Is the maximum number of pods deployable to a node configurable?

    是,當您使用 Azure CLI 或 Resource Manager 範本部署叢集時。Yes, when you deploy a cluster with the Azure CLI or a Resource Manager template. 請參閱每個節點的最大 Pod 數目See Maximum pods per node.

    您無法在現叢集上變更每個節點的 Pod 數目上限。You can't change the maximum number of pods per node on an existing cluster.

  • 如何針對在 AKS 叢集建立期間所建立的子網路設定其他屬性?例如,服務端點。How do I configure additional properties for the subnet that I created during AKS cluster creation? For example, service endpoints.

    在 AKS 叢集建立期間所建立的虛擬網路和子網路屬性完整清單,均可在 Azure 入口網站的標準虛擬網路組態頁面中進行設定。The complete list of properties for the virtual network and subnets that you create during AKS cluster creation can be configured in the standard virtual network configuration page in the Azure portal.

  • 可以使用叢集虛擬網路內的不同子網路作為 Kubernetes 服務位址範圍嗎?Can I use a different subnet within my cluster virtual network for the Kubernetes service address range?

    不建議,但此組態是可行的。It's not recommended, but this configuration is possible. 服務位址範圍是 Kubernetes 指派給您叢集中內部服務的一組虛擬 IP (VIP)。The service address range is a set of virtual IPs (VIPs) that Kubernetes assigns to internal services in your cluster. Azure 網路功能無法查看 Kubernetes 叢集的服務 IP 範圍。Azure Networking has no visibility into the service IP range of the Kubernetes cluster. 因為無法查看叢集的服務位址範圍,所以稍後有可能在與服務位址範圍重疊的叢集虛擬網路中建立新的子網路。Because of the lack of visibility into the cluster's service address range, it's possible to later create a new subnet in the cluster virtual network that overlaps with the service address range. 如果發生這類重疊,Kubernetes 可能會將子網路中另一項資源已經使用的 IP 指派給服務,因而造成無法預期的行為或失敗。If such an overlap occurs, Kubernetes could assign a service an IP that's already in use by another resource in the subnet, causing unpredictable behavior or failures. 您可藉由確保您使用叢集虛擬網路外部的位址範圍,避免此重疊風險。By ensuring you use an address range outside the cluster's virtual network, you can avoid this overlap risk.

後續步驟Next steps

在下列文章中深入了解 AKS 的網路功能:Learn more about networking in AKS in the following articles:

AKS 引擎AKS Engine

Azure Kubernetes Service 引擎 (AKS engine)是一個開放原始碼專案, 會產生 Azure Resource Manager 範本, 您可以用來在 Azure 上部署 Kubernetes 叢集。Azure Kubernetes Service Engine (AKS Engine) is an open-source project that generates Azure Resource Manager templates you can use for deploying Kubernetes clusters on Azure.

使用 AKS 引擎建立的 Kubernetes 叢集同時支援kubenetAzure CNI外掛程式。Kubernetes clusters created with AKS Engine support both the kubenet and Azure CNI plugins. 因此,這兩個網路案例都會受到 AKS 引擎支援。As such, both networking scenarios are supported by AKS Engine.