在 Azure Kubernetes Service (AKS) 中使用 kubenet 網路與您自己的 IP 位址範圍Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS)

根據預設, AKS 叢集會使用kubenet, 並為您建立 Azure 虛擬網路和子網。By default, AKS clusters use kubenet, and an Azure virtual network and subnet are created for you. 使用 kubenet,節點會從 Azure 虛擬網路子網路取得 IP 位址。With kubenet, nodes get an IP address from the Azure virtual network subnet. Pod 會從邏輯上不同的位址空間接收至節點的 Azure 虛擬網路子網路的 IP 位址。Pods receive an IP address from a logically different address space to the Azure virtual network subnet of the nodes. 然後設定網路位址轉譯 (NAT),以便 Pod 可以連線到 Azure 虛擬網路上的資源。Network address translation (NAT) is then configured so that the pods can reach resources on the Azure virtual network. 流量的來源 IP 位址是 NAT 到節點的主要 IP 位址。The source IP address of the traffic is NAT'd to the node's primary IP address. 這種方法可大幅減少您需要在網路空間中保留,以供 Pod 使用的 IP 位址數目。This approach greatly reduces the number of IP addresses that you need to reserve in your network space for pods to use.

使用Azure 容器網路介面 (CNI), 每個 pod 都會從子網取得 IP 位址, 而且可以直接存取。With Azure Container Networking Interface (CNI), every pod gets an IP address from the subnet and can be accessed directly. 這些 IP 位址在您的網路空間中必須是唯一的,且必須事先規劃。These IP addresses must be unique across your network space, and must be planned in advance. 每個節點都有一個組態參數,用於所支援的最大 Pod 數目。Each node has a configuration parameter for the maximum number of pods that it supports. 然後,為該節點預先保留每個節點的相同 IP 位址數目。The equivalent number of IP addresses per node are then reserved up front for that node. 此方法需要更多的規劃,並且通常會導致 IP 位址耗盡,或者隨著應用程式需求增加,需要在更大的子網路中重建叢集。This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow.

本文將說明如何使用 kubenet 網路來建立虛擬網路子網路,並將其與 AKS 叢集搭配使用。This article shows you how to use kubenet networking to create and use a virtual network subnet for an AKS cluster. 如需網路選項和考慮的詳細資訊, 請參閱Kubernetes 和 AKS 的網路概念For more information on network options and considerations, see Network concepts for Kubernetes and AKS.

警告

若要使用 Windows Server 節點集區 (目前在 AKS 中處於預覽狀態), 您必須使用 Azure CNI。To use Windows Server node pools (currently in preview in AKS), you must use Azure CNI. 使用 kubenet 作為網路模型不適用於 Windows Server 容器。The use of kubenet as the network model is not available for Windows Server containers.

開始之前Before you begin

您需要安裝並設定 Azure CLI 版本2.0.65 或更新版本。You need the Azure CLI version 2.0.65 or later installed and configured. 執行  az --version 以尋找版本。Run az --version to find the version. 如果您需要安裝或升級, 請參閱 安裝 Azure CLIIf you need to install or upgrade, see Install Azure CLI.

使用您自己的子網路的 Kubenet 網路概觀Overview of kubenet networking with your own subnet

在許多環境中,您已定義具有已配置的 IP 位址範圍的虛擬網路和子網路。In many environments, you have defined virtual networks and subnets with allocated IP address ranges. 這些虛擬網路資源用來支援多個服務和應用程式。These virtual network resources are used to support multiple services and applications. 若要提供網路連線,AKS 叢集可以使用 kubenet (基本網路) 或 Azure CNI (進階網路)。To provide network connectivity, AKS clusters can use kubenet (basic networking) or Azure CNI (advanced networking).

使用 kubenet,只有節點在虛擬網路子網路中接收 IP 位址。With kubenet, only the nodes receive an IP address in the virtual network subnet. Pod 無法彼此直接通訊。Pods can't communicate directly with each other. 相反地,使用者定義路由 (UDR) 和 IP 轉送會用來進行 跨節點的 Pod 之間的連線。Instead, User Defined Routing (UDR) and IP forwarding is used for connectivity between pods across nodes. 您還可以在接收已指派 IP 位址的服務後面部署 Pod,並為應用程式的流量進行負載平衡。You could also deploy pods behind a service that receives an assigned IP address and load balances traffic for the application. 下圖顯示 AKS 節點如何在虛擬網路子網路中 (而不是 Pod 中) 接收 IP 位址:The following diagram shows how the AKS nodes receive an IP address in the virtual network subnet, but not the pods:

Kubenet 網路模型與 AKS 叢集

Azure 在 UDR 中最多支援 400 條路由,因此您不能擁有超過 400 個節點的 AKS 叢集。Azure supports a maximum of 400 routes in a UDR, so you can't have an AKS cluster larger than 400 nodes. Kubenet不支援 AKS 功能, 例如虛擬節點或網路原則。AKS features such as Virtual Nodes or network policies aren't supported with kubenet.

使用 Azure CNI,每個 Pod 都會接收 IP 子網路中的 IP 位址,並可以直接與其他 Pod 和服務進行通訊。With Azure CNI, each pod receives an IP address in the IP subnet, and can directly communicate with other pods and services. 您的叢集可以與您指定的 IP 位址範圍一樣大。Your clusters can be as large as the IP address range you specify. 不過,必須事先規劃 IP 位址範圍,並且 AKS 節點根據它們可以支援的最大 Pod 數目來使用所有 IP 位址。However, the IP address range must be planned in advance, and all of the IP addresses are consumed by the AKS nodes based on the maximum number of pods that they can support. AZURE CNI支援先進的網路功能和案例, 例如虛擬節點或網路原則。Advanced network features and scenarios such as Virtual Nodes or network policies are supported with Azure CNI.

IP 位址可用性與耗盡IP address availability and exhaustion

對於 Azure CNI,常見問題是指派的 IP 位址範圍太小,無法在調整或將叢集升級時加入其他節點。With Azure CNI, a common issue is the assigned IP address range is too small to then add additional nodes when you scale or upgrade a cluster. 網路小組可能也無法發出夠大的 IP 位址範圍來支援您預期的應用程式需求。The network team may also not be able to issue a large enough IP address range to support your expected application demands.

作為折衷方案,您可以建立使用 kubenet 並連接到現有虛擬網路子網路的 AKS 叢集。As a compromise, you can create an AKS cluster that uses kubenet and connect to an existing virtual network subnet. 此方法可讓節點接收定義的 IP 位址,而無需事先為叢集中可能執行的所有潛在 Pod 保留大量的 IP 位址。This approach lets the nodes receive defined IP addresses, without the need to reserve a large number of IP addresses up front for all of the potential pods that could run in the cluster.

使用 kubenet,您可以使用更小的 IP 位址範圍,並能夠支援大型叢集和應用程式需求。With kubenet, you can use a much smaller IP address range and be able to support large clusters and application demands. 例如,即使具有 /27 IP 位址範圍,您也可以執行具有足夠擴充或升級空間的 20-25 節點叢集。For example, even with a /27 IP address range, you could run a 20-25 node cluster with enough room to scale or upgrade. 此叢集大小可支援最多 2,200-2,750 個 Pod (每個節點預設最多 110 個 Pod)。This cluster size would support up to 2,200-2,750 pods (with a default maximum of 110 pods per node). 您可以使用 AKS 中的kubenet設定的每個節點的 pod 數目上限為110。The maximum number of pods per node that you can configure with kubenet in AKS is 110.

下列基本計算會比較網路模型中的差異:The following basic calculations compare the difference in network models:

  • kubenet - 一個簡單的 /24 IP 位址範圍,最多可支援叢集中的 251 個節點 (每個 Azure 虛擬網路子網路會保留前三個用於管理作業的 IP 位址)kubenet - a simple /24 IP address range can support up to 251 nodes in the cluster (each Azure virtual network subnet reserves the first three IP addresses for management operations)
    • 此節點計數最多可以支援 27,610 個 Pod (kubenet 每個節點預設最多 110 個 Pod)This node count could support up to 27,610 pods (with a default maximum of 110 pods per node with kubenet)
  • Azure CNI - 相同的基本 /24 子網路範圍只能支援叢集中最多 8 個節點Azure CNI - that same basic /24 subnet range could only support a maximum of 8 nodes in the cluster
    • 此節點計數最多僅支援 240 個 Pod (Azure CNI 每個節點預設最多 30 個 Pod)This node count could only support up to 240 pods (with a default maximum of 30 pods per node with Azure CNI)

注意

這些最大值不列入帳戶升級,或調整規模作業。These maximums don't take into account upgrade or scale operations. 實際上,您無法執行子網路 IP 位址範圍所支援的節點數目上限。In practice, you can't run the maximum number of nodes that the subnet IP address range supports. 您必須在升級作業期間保留一些可用的 IP 位址。You must leave some IP addresses available for use during scale of upgrade operations.

虛擬網路對等互連和 ExpressRoute 連線Virtual network peering and ExpressRoute connections

為了提供內部部署連線能力, kubenetAzure CNI網路方法都可以使用Azure 虛擬網路對等互連ExpressRoute連線。To provide on-premises connectivity, both kubenet and Azure-CNI network approaches can use Azure virtual network peering or ExpressRoute connections. 仔細規劃您的 IP 位址範圍,以避免重疊和不正確的流量路由。Plan your IP address ranges carefully to prevent overlap and incorrect traffic routing. 例如,許多內部部署網路使用透過 ExpressRoute 連線通告的 10.0.0.0/8 位址範圍。For example, many on-premises networks use a 10.0.0.0/8 address range that is advertised over the ExpressRoute connection. 建議您在此位址範圍外的 Azure 虛擬網路子網中建立 AKS 叢集, 例如172.16.0.0/16It's recommended to create your AKS clusters into Azure virtual network subnets outside of this address range, such as 172.16.0.0/16.

選擇要使用的網路模型Choose a network model to use

選擇用於AKS群集的網絡插件通常是靈活性和高級配置需求之間的平衡。The choice of which network plugin to use for your AKS cluster is usually a balance between flexibility and advanced configuration needs. 下列考量有助於概述每種網路模型最合適的時機。The following considerations help outline when each network model may be the most appropriate.

kubenet 使用時機:Use kubenet when:

  • 您的 IP 位址空間有限。You have limited IP address space.
  • 大部分的 Pod 通訊是在叢集內。Most of the pod communication is within the cluster.
  • 您不需要虛擬節點或網路原則等進階功能。You don't need advanced features such as virtual nodes or network policy.

Azure CNI 使用時機:Use Azure CNI when:

  • 您有可用的 IP 位址空間。You have available IP address space.
  • 大部分的 Pod 通訊是在叢集外部的資源。Most of the pod communication is to resources outside of the cluster.
  • 您不想管理 UDR。You don’t want to manage the UDRs.
  • 您需要進階功能,例如虛擬節點或網路原則。You need advanced features such as virtual nodes or network policy.

如需可協助您決定要使用哪一個網路模型的詳細資訊, 請參閱比較網路模型及其支援範圍For more information to help you decide which network model to use, see Compare network models and their support scope.

注意

Kuberouter 可讓您在使用 kubenet 時啟用網路原則, 而且可以在 AKS 叢集中安裝為 daemonset。Kuberouter makes it possible to enable network policy when using kubenet and can be installed as a daemonset in an AKS cluster. 請注意 kube-路由器仍在搶鮮版 (Beta), 而 Microsoft 不提供專案的任何支援。Please be aware kube-router is still in beta and no support is offered by Microsoft for the project.

建立虛擬網路和子網路Create a virtual network and subnet

若要開始使用kubenet和您自己的虛擬網路子網, 請先使用az group create命令來建立資源群組。To get started with using kubenet and your own virtual network subnet, first create a resource group using the az group create command. 下列範例會在 eastus 位置建立名為 myResourceGroup 的資源群組:The following example creates a resource group named myResourceGroup in the eastus location:

az group create --name myResourceGroup --location eastus

如果您沒有現有的虛擬網路和子網可供使用, 請使用az network vnet create命令來建立這些網路資源。If you don't have an existing virtual network and subnet to use, create these network resources using the az network vnet create command. 在下列範例中, 會將虛擬網路命名為myVnet , 位址首碼為192.168.0.0/16In the following example, the virtual network is named myVnet with the address prefix of 192.168.0.0/16. 建立名為myAKSSubnet的子網, 位址首碼為192.168.1.0/24A subnet is created named myAKSSubnet with the address prefix 192.168.1.0/24.

az network vnet create \
    --resource-group myResourceGroup \
    --name myAKSVnet \
    --address-prefixes 192.168.0.0/16 \
    --subnet-name myAKSSubnet \
    --subnet-prefix 192.168.1.0/24

建立服務主體並指派權限Create a service principal and assign permissions

為了允許 AKS 叢集與其他 Azure 資源互動,則會使用 Azure Active Directory 服務主體。To allow an AKS cluster to interact with other Azure resources, an Azure Active Directory service principal is used. 服務主體必須具有管理 AKS 節點使用的虛擬網路和子網路的權限。The service principal needs to have permissions to manage the virtual network and subnet that the AKS nodes use. 若要建立服務主體, 請使用az ad sp create-rbac命令:To create a service principal, use the az ad sp create-for-rbac command:

az ad sp create-for-rbac --skip-assignment

下列範例輸出顯示您的服務主體的應用程式識別碼和密碼。The following example output shows the application ID and password for your service principal. 這些值用於其他步驟,以將角色指派給服務主體,然後再建立 AKS 叢集:These values are used in additional steps to assign a role to the service principal and then create the AKS cluster:

$ az ad sp create-for-rbac --skip-assignment

{
  "appId": "476b3636-5eda-4c0e-9751-849e70b5cfad",
  "displayName": "azure-cli-2019-01-09-22-29-24",
  "name": "http://azure-cli-2019-01-09-22-29-24",
  "password": "a1024cd7-af7b-469f-8fd7-b293ecbb174e",
  "tenant": "72f998bf-85f1-41cf-92ab-2e7cd014db46"
}

若要在其餘步驟中指派正確的委派, 請使用az network vnet showaz network vnet subnet show命令來取得所需的資源識別碼。To assign the correct delegations in the remaining steps, use the az network vnet show and az network vnet subnet show commands to get the required resource IDs. 這些資源識別碼會儲存為變數,並在其餘的步驟中參考:These resource IDs are stored as variables and referenced in the remaining steps:

VNET_ID=$(az network vnet show --resource-group myResourceGroup --name myAKSVnet --query id -o tsv)
SUBNET_ID=$(az network vnet subnet show --resource-group myResourceGroup --vnet-name myAKSVnet --name myAKSSubnet --query id -o tsv)

現在, 使用az role 指派 create命令, 為您的 AKS 叢集參與者許可權指派虛擬網路的服務主體。Now assign the service principal for your AKS cluster Contributor permissions on the virtual network using the az role assignment create command. 提供您自己 <的 appId > , 如先前命令的輸出中所示, 以建立服務主體:Provide your own <appId> as shown in the output from the previous command to create the service principal:

az role assignment create --assignee <appId> --scope $VNET_ID --role Contributor

在虛擬網路中建立 AKS 叢集Create an AKS cluster in the virtual network

您現在已經建立虛擬網路和子網路,並為服務主體建立並指派使用這些網路資源的權限。You've now created a virtual network and subnet, and created and assigned permissions for a service principal to use those network resources. 現在, 使用az AKS create命令, 在您的虛擬網路和子網中建立 AKS 叢集。Now create an AKS cluster in your virtual network and subnet using the az aks create command. 定義您自己的服務主體 <appId ><密碼 > , 如先前命令的輸出中所示, 以建立服務主體。Define your own service principal <appId> and <password>, as shown in the output from the previous command to create the service principal.

下列 IP 位址範圍也定義為叢集建立程序的一部分:The following IP address ranges are also defined as part of the cluster create process:

  • --service-cidr 用於為 AKS 叢集中的內部服務指派 IP 位址。The --service-cidr is used to assign internal services in the AKS cluster an IP address. 此 IP 位址範圍應該是您的網路環境中其他未使用的位址空間。This IP address range should be an address space that isn't in use elsewhere in your network environment. 如果您使用 Express Route 或站對站 VPN 連線來連線或規劃連接 Azure 虛擬網路, 此範圍會包含任何內部部署網路範圍。This range includes any on-premises network ranges if you connect, or plan to connect, your Azure virtual networks using Express Route or a Site-to-Site VPN connection.

  • --dns-service-ip 位址應該是服務 IP 位址範圍的 .10 位址。The --dns-service-ip address should be the .10 address of your service IP address range.

  • --pod-cidr 應該是您的網路環境中未使用的大型位址空間。The --pod-cidr should be a large address space that isn't in use elsewhere in your network environment. 如果您使用 Express Route 或站對站 VPN 連線來連線或規劃連接 Azure 虛擬網路, 此範圍會包含任何內部部署網路範圍。This range includes any on-premises network ranges if you connect, or plan to connect, your Azure virtual networks using Express Route or a Site-to-Site VPN connection.

    • 此位址範圍必須大到足以容納您希望相應增加的節點數目。This address range must be large enough to accommodate the number of nodes that you expect to scale up to. 如果您需要更多位址用於其他節點,則無法在部署叢集之後變更此位址範圍。You can't change this address range once the cluster is deployed if you need more addresses for additional nodes.
    • Pod IP 位址範圍用來為叢集中的每個節點指派 /24 位址空間。The pod IP address range is used to assign a /24 address space to each node in the cluster. 在下列範例中, 10.244.0.0/16--pod-cidr會指派第一個節點10.244.0.0/24、第二個節點10.244.1.0/24和第三個節點10.244.2.0/24In the following example, the --pod-cidr of 10.244.0.0/16 assigns the first node 10.244.0.0/24, the second node 10.244.1.0/24, and the third node 10.244.2.0/24.
    • 隨著叢集縮放比例或升級,Azure 平台會繼續為每個新的節點指派一個 Pod IP 位址範圍。As the cluster scales or upgrades, the Azure platform continues to assign a pod IP address range to each new node.
  • --Docker-橋接器位址可讓 AKS 節點與基礎管理平臺進行通訊。The --docker-bridge-address lets the AKS nodes communicate with the underlying management platform. 此 IP 位址不能在您叢集的虛擬網路 IP 位址範圍內,而且不應該與您網路上使用中的其他位址範圍重疊。This IP address must not be within the virtual network IP address range of your cluster, and shouldn't overlap with other address ranges in use on your network.

az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --node-count 3 \
    --network-plugin kubenet \
    --service-cidr 10.0.0.0/16 \
    --dns-service-ip 10.0.0.10 \
    --pod-cidr 10.244.0.0/16 \
    --docker-bridge-address 172.17.0.1/16 \
    --vnet-subnet-id $SUBNET_ID \
    --service-principal <appId> \
    --client-secret <password>

當您建立 AKS 叢集時,會建立網路安全群組和路由表。When you create an AKS cluster, a network security group and route table are created. 這些網路資源是由 AKS 控制平面所管理。These network resources are managed by the AKS control plane. 網路安全性群組會自動與您節點上的虛擬 Nic 相關聯。The network security group is automatically associated with the virtual NICs on your nodes. 路由表會自動與虛擬網路子網建立關聯。The route table is automatically associated with the virtual network subnet. 當您建立和公開服務時, 網路安全性群組規則和路由表和會自動更新。Network security group rules and route tables and are automatically updated as you create and expose services.

後續步驟Next steps

透過將 AKS 叢集部署到您現有的虛擬網路子網路中,您現在可以如往常一樣使用叢集。With an AKS cluster deployed into your existing virtual network subnet, you can now use the cluster as normal. 開始使用 Azure Dev Spaces 或使用草稿建立應用程式, 或使用Helm 部署應用程式Get started with building apps using Azure Dev Spaces or using Draft, or deploy apps using Helm.