Azure Kubernetes Service (AKS) 中的網路連線和安全性最佳做法Best practices for network connectivity and security in Azure Kubernetes Service (AKS)

當您在 Azure Kubernetes Service (AKS) 中建立及管理叢集時,您會為節點和應用程式提供網路連線。As you create and manage clusters in Azure Kubernetes Service (AKS), you provide network connectivity for your nodes and applications. 這些網路資源包括 IP 位址範圍、負載平衡器和輸入控制器。These network resources include IP address ranges, load balancers, and ingress controllers. 若要維持高品質的應用程式服務,您需要規劃並設定這些資源。To maintain a high quality of service for your applications, you need to plan for and then configure these resources.

此最佳做法文章著重於叢集操作員的網路連線能力和安全性。This best practices article focuses on network connectivity and security for cluster operators. 在本文中,您將了解:In this article, you learn how to:

  • 比較 AKS 中的 Kubenet 和 Azure CNI 網路模式Compare the kubenet and Azure CNI network modes in AKS
  • 規劃所需的 IP 位址和連線能力Plan for required IP addressing and connectivity
  • 使用負載平衡器、輸入控制器或 Web 應用程式防火牆 (WAF) 來分配流量Distribute traffic using load balancers, ingress controllers, or a web application firewalls (WAF)
  • 安全地連線到叢集節點Securely connect to cluster nodes

選擇適當的網路模型Choose the appropriate network model

最佳做法指引 - 若要整合現有虛擬網路或內部部署網路,請使用 AKS 中的 Azure CNI 網路功能。Best practice guidance - For integration with existing virtual networks or on-premises networks, use Azure CNI networking in AKS. 此網路模型也可讓企業環境中的資源和控制項達到更有效的區隔。This network model also allows greater separation of resources and controls in an enterprise environment.

虛擬網路會提供存取您應用程式的基本連線能力給 AKS 節點和客戶。Virtual networks provide the basic connectivity for AKS nodes and customers to access your applications. 將 AKS 叢集部署到虛擬網路有兩種不同的方式:There are two different ways to deploy AKS clusters into virtual networks:

  • 網路 Kubenet -Azure 管理的虛擬網路資源的叢集部署,並使用kubenet Kubernetes 外掛程式。Kubenet networking - Azure manages the virtual network resources as the cluster is deployed and uses the kubenet Kubernetes plugin.
  • Azure CNI 網路-部署到現有的虛擬網路,並使用Azure 容器網路介面 (CNI) Kubernetes 外掛程式。Azure CNI networking - Deploys into an existing virtual network, and uses the Azure Container Networking Interface (CNI) Kubernetes plugin. Pod 會收到可路由到其他網路服務或內部部署資源的個別 IP。Pods receive individual IPs that can route to other network services or on-premises resources.

容器網路介面 (CNI) 是一個「廠商中立」通訊協定,可讓容器執行階段對網路提供者提出要求。The Container Networking Interface (CNI) is a vendor-neutral protocol that lets the container runtime make requests to a network provider. Azure CNI 會將 IP 位址指派給 Pod 和節點,並在您連線至現有 Azure 虛擬網路時提供 IP 位址管理 (IPAM) 功能。The Azure CNI assigns IP addresses to pods and nodes, and provides IP address management (IPAM) features as you connect to existing Azure virtual networks. 每個節點和 Pod 資源都會收到 Azure 虛擬網路中的 IP 位址,並且不需要使用額外路由來與其他資源或服務通訊。Each node and pod resource receives an IP address in the Azure virtual network, and no additional routing is needed to communicate with other resources or services.

此圖表顯示兩個節點,且各有橋接器將其連線至單一 Azure VNet

您應對大部分的生產環境部署使用 Azure CNI 網路功能。For most production deployments, you should use Azure CNI networking. 此網路模型可用來區隔資源的控制和管理。This network model allows for separation of control and management of resources. 從安全性觀點來看,您通常會想讓不同小組來管理及保護這些資源。From a security perspective, you often want different teams to manage and secure those resources. Azure CNI 網路功能可讓您透過指派給每個 Pod 的 IP 位址,直接連線到現有的 Azure 資源、內部部署資源或其他服務。Azure CNI networking lets you connect to existing Azure resources, on-premises resources, or other services directly via IP addresses assigned to each pod.

當您使用 Azure CNI 網路功能時,虛擬網路資源會在 AKS 叢集的不同資源群組中。When you use Azure CNI networking, the virtual network resource is in a separate resource group to the AKS cluster. 您可以將存取和管理這些資源的權限委派給 AKS 服務主體。Delegate permissions for the AKS service principal to access and manage these resources. AKS 叢集所使用的服務主體在您虛擬網路內的子網路上必須至少具有網路參與者權限。The service principal used by the AKS cluster must have at least Network Contributor permissions on the subnet within your virtual network. 如果您想要定義自訂角色,而不使用內建的網路參與者角色,則需要下列權限:If you wish to define a custom role instead of using the built-in Network Contributor role, the following permissions are required:

  • Microsoft.Network/virtualNetworks/subnets/join/action
  • Microsoft.Network/virtualNetworks/subnets/read

如需有關 AKS 服務主體委派的詳細資訊,請參閱存取權委派給其他 Azure 資源For more information about AKS service principal delegation, see Delegate access to other Azure resources.

當每個節點和 Pod 收到自己的 IP 位址時,請規劃 AKS 子網路的位址範圍。As each node and pod receive its own IP address, plan out the address ranges for the AKS subnets. 子網路必須夠大,才能為您部署的每個節點、Pod 和網路資源提供 IP 位址。The subnet must be large enough to provide IP addresses for every node, pods, and network resources that you deploy. 每個 AKS 叢集必須放在自己的子網路中。Each AKS cluster must be placed in its own subnet. 若要在 Azure 中允許對內部部署或對等互連網路進行連線,請不要使用與現有網路資源重疊的 IP 位址範圍。To allow connectivity to on-premises or peered networks in Azure, don't use IP address ranges that overlap with existing network resources. Kubenet 和 Azure CNI 網路功能都有預設每個節點可執行的 Pod 數目限制。There are default limits to the number of pods that each node runs with both kubenet and Azure CNI networking. 若要處理相應增加事件或叢集升級,您也需要其他可在所指派子網路中使用的 IP 位址。To handle scale up events or cluster upgrades, you also need additional IP addresses available for use in the assigned subnet. 這個額外的位址空間是特別重要,如果您將使用 Windows Server 容器 (目前處於預覽 AKS 中),因為這些節點的集區需要升級至套用最新的安全性修補程式。This additional address space is especially important if you use Windows Server containers (currently in preview in AKS), as those node pools require an upgrade to apply the latest security patches. 如需有關 Windows 伺服器節點的詳細資訊,請參閱升級 AKS 中的節點集區For more information on Windows Server nodes, see Upgrade a node pool in AKS.

若要計算的 IP 位址有需要,請參閱AKS 中的設定 Azure CNI 網路To calculate the IP address required, see Configure Azure CNI networking in AKS.

Kubenet 網路功能Kubenet networking

雖然 Kubenet 不需要在部署叢集之前先設定虛擬網路,但也有些缺點:Although kubenet doesn't require you to set up the virtual networks before the cluster is deployed, there are disadvantages:

  • 節點和 Pod 放置於不同的 IP 子網路。Nodes and pods are placed on different IP subnets. 使用者定義路由 (UDR) 和 IP 轉送會用來路由 Pod 和節點之間的流量。User Defined Routing (UDR) and IP forwarding is used to route traffic between pods and nodes. 這個額外的路由會降低網路效能。This additional routing may reduce network performance.
  • 連線至現有內部部署網路或與其他 Azure 虛擬網路對等互連可能會很複雜。Connections to existing on-premises networks or peering to other Azure virtual networks can be complex.

Kubenet 適用於小型的開發或測試工作負載,因為您不需要從 AKS 叢集分別建立虛擬網路和子網路。Kubenet is suitable for small development or test workloads, as you don't have to create the virtual network and subnets separately from the AKS cluster. 使用 Kubenet 網路功能部署的簡單 AKS 叢集適用於流量低的簡易網站,或用於將工作負載隨即轉移到容器中。Simple websites with low traffic, or to lift and shift workloads into containers, can also benefit from the simplicity of AKS clusters deployed with kubenet networking. 您應對大部分的生產環境部署規劃並使用 Azure CNI 網路功能。For most production deployments, you should plan for and use Azure CNI networking. 您也可以設定您自己的 IP 位址範圍和虛擬網路使用 kubenetYou can also configure your own IP address ranges and virtual networks using kubenet.

分配輸入流量Distribute ingress traffic

最佳做法指引 - 若要將 HTTP 或 HTTPS 流量分散到應用程式,請使用輸入資源和控制器。Best practice guidance - To distribute HTTP or HTTPS traffic to your applications, use ingress resources and controllers. 輸入控制器會透過一般的 Azure 負載平衡器來提供額外功能,並可作為原生 Kubernetes 資源來加以管理。Ingress controllers provide additional features over a regular Azure load balancer, and can be managed as native Kubernetes resources.

Azure 負載平衡器可以將客戶流量分散到 AKS 叢集中的應用程式,但會受限於應用程式對該流量的了解程度。An Azure load balancer can distribute customer traffic to applications in your AKS cluster, but it's limited in what it understands about that traffic. 負載平衡器資源會在第 4 層上運作,並根據通訊協定或連接埠分散流量。A load balancer resource works at layer 4, and distributes traffic based on protocol or ports. 大部分使用 HTTP 或 HTTPS 的 Web 應用程式應使用在第 7 層運作的 Kuberenetes 輸入資源和控制器。Most web applications that use HTTP or HTTPS should use Kuberenetes ingress resources and controllers, which work at layer 7. 輸入可以根據應用程式的 URL 將流量分散到應用程式,並處理 TLS/SSL 終止。Ingress can distribute traffic based on the URL of the application and handle TLS/SSL termination. 這項功能也會減少您公開並對應的 IP 位址數目。This ability also reduces the number of IP addresses you expose and map. 若使用負載平衡器,每個應用程式通常需要指派並對應至 AKS 叢集中服務的公用 IP 位址。With a load balancer, each application typically needs a public IP address assigned and mapped to the service in the AKS cluster. 若使用輸入資源,單一 IP 位址可以將流量分散到多個應用程式。With an ingress resource, a single IP address can distribute traffic to multiple applications.

此圖顯示 AKS 叢集中的輸入流量

輸入有兩個元件:There are two components for ingress:

  • 輸入「資源」 ,以及An ingress resource, and
  • 輸入「控制器」 An ingress controller

輸入資源是 kind: Ingress 的 YAML資訊清單,用來定義主機、憑證及規則,以將流量路由至在您 AKS 叢集中執行的服務。The ingress resource is a YAML manifest of kind: Ingress that defines the host, certificates, and rules to route traffic to services that run in your AKS cluster. 下列 YAML 資訊清單範例會將 myapp.com 的流量分散到以下兩個服務的其中一個:blogservice 或 storeservice 。The following example YAML manifest would distribute traffic for myapp.com to one of two services, blogservice or storeservice. 系統會根據客戶存取的 URL,將他們導向其中一個服務。The customer is directed to one service or the other based on the URL they access.

kind: Ingress
metadata:
 name: myapp-ingress
   annotations: kubernetes.io/ingress.class: "PublicIngress"
spec:
 tls:
 - hosts:
   - myapp.com
   secretName: myapp-secret
 rules:
   - host: myapp.com
     http:
      paths:
      - path: /blog
        backend:
         serviceName: blogservice
         servicePort: 80
      - path: /store
        backend:
         serviceName: storeservice
         servicePort: 80

輸入控制器是在 AKS 節點上執行的精靈,可監控傳入要求。An ingress controller is a daemon that runs on an AKS node and watches for incoming requests. 接著,流量會根據輸入資源中所定義的規則來分配。Traffic is then distributed based on the rules defined in the ingress resource. 最常見的輸入控制器是以 NGINX 為基礎。The most common ingress controller is based on NGINX. AKS 並不會限制您特定的控制站,因此您可以使用其他控制站,例如Contour, HAProxy,或TraefikAKS doesn't restrict you to a specific controller, so you can use other controllers such as Contour, HAProxy, or Traefik.

輸入控制器必須在 Linux 節點上排程。Ingress controllers must be scheduled on a Linux node. Windows Server (目前在 AKS 中的預覽) 的節點不應該執行輸入控制器。Windows Server nodes (currently in preview in AKS) shouldn't run the ingress controller. 您可以使用 YAML 資訊清單或 Helm 圖表部署中的節點選取器,指出,在以 Linux 為基礎的節點上執行的資源。Use a node selector in your YAML manifest or Helm chart deployment to indicate that the resource should run on a Linux-based node. 如需詳細資訊,請參閱 < 使用 AKS 中排程 pod 的所在位置的節點選取器,來控制For more information, see Use node selectors to control where pods are scheduled in AKS.

有許多適用輸入的案例,包括下列的使用說明指南:There are many scenarios for ingress, including the following how-to guides:

使用 Web 應用程式防火牆 (WAF) 來保護流量Secure traffic with a web application firewall (WAF)

最佳作法指引-若要掃描潛在攻擊的連入流量,請使用 web 應用程式防火牆 (WAF),例如Barracuda WAF for Azure或 Azure 應用程式閘道。Best practice guidance - To scan incoming traffic for potential attacks, use a web application firewall (WAF) such as Barracuda WAF for Azure or Azure Application Gateway. 這些更進階的網路資源也可以路由流量,並非僅限 HTTP 和 HTTPS 連線或是基本 SSL 終止。These more advanced network resources can also route traffic beyond just HTTP and HTTPS connections or basic SSL termination.

將流量分散到服務和應用程式的輸入控制器通常是您 AKS 叢集中的 Kubernetes 資源。An ingress controller that distributes traffic to services and applications is typically a Kubernetes resource in your AKS cluster. 控制器會以精靈形式執行於 AKS 節點上,並取用一些節點的資源,例如 CPU、記憶體和網路頻寬。The controller runs as a daemon on an AKS node, and consumes some of the node's resources such as CPU, memory, and network bandwidth. 在較大型的環境中,您通常會想要卸載一些此流量路由或 TLS 終止至 AKS 叢集外部的網路資源。In larger environments, you often want to offload some of this traffic routing or TLS termination to a network resource outside of the AKS cluster. 並且想掃描傳入的流量以查看是否有潛在攻擊。You also want to scan incoming traffic for potential attacks.

Azure 應用程式閘道等 Web 應用程式防火牆 (WAF) 可以保護並分散您 AKS 叢集的流量

Web 應用程式防火牆 (WAF) 會藉由篩選傳入流量來提供額外一層安全性。A web application firewall (WAF) provides an additional layer of security by filtering the incoming traffic. Open Web Application Security Project (OWASP) 會提供一組規則來監看是否有跨網站指令碼或 Cookie 篡改等攻擊。The Open Web Application Security Project (OWASP) provides a set of rules to watch for attacks like cross site scripting or cookie poisoning. Azure 應用程式閘道(目前為預覽狀態在 AKS 中的) 是可以與 AKS 叢集,以提供這些安全性功能,才能將流量到達您的 AKS 叢集與應用程式整合 WAF。Azure Application Gateway (currently in preview in AKS) is a WAF that can integrate with AKS clusters to provide these security features, before the traffic reaches your AKS cluster and applications. 其他第三方解決方案也會執行這些功能,因此您可以在指定產品中繼續使用現有的投資或專業技術。Other third-party solutions also perform these functions, so you can continue to use existing investments or expertise in a given product.

負載平衡器或輸入資源會繼續在您的 AKS 叢集執行,以進一步精簡流量分配。Load balancer or ingress resources continue to run in your AKS cluster to further refine the traffic distribution. 您可以使用資源定義,將應用程式閘道當作輸入控制器來集中管理。App Gateway can be centrally managed as an ingress controller with a resource definition. 若要開始,建立應用程式閘道輸入控制器To get started, create an Application Gateway Ingress controller.

使用網路原則控制流量流程Control traffic flow with network policies

最佳作法指引 - 使用網路原則允許或拒絕 Pod 的流量。Best practice guidance - Use network policies to allow or deny traffic to pods. 根據預設,叢集中的 Pod 之間允許所有流量。By default, all traffic is allowed between pods within a cluster. 為了提升安全性,請定義限制 Pod 通訊的規則。For improved security, define rules that limit pod communication.

網路原則是一種 Kubernetes 功能,可讓您控制 Pod 之間的流量。Network policy is a Kubernetes feature that lets you control the traffic flow between pods. 您可以根據指派的標籤、命名空間或流量連接埠等設定,選擇允許或拒絕流量。You can choose to allow or deny traffic based on settings such as assigned labels, namespace, or traffic port. 使用網路原則提供了一種雲端原生方法來控制流量的流程。The use of network policies gives a cloud-native way to control the flow of traffic. 由於在 AKS 叢集中動態建立 Pod,因此可以自動套用所需的網路原則。As pods are dynamically created in an AKS cluster, the required network policies can be automatically applied. 請勿使用 Azure 網路安全性群組來控制 pod-to-pod 流量,請使用網路原則。Don't use Azure network security groups to control pod-to-pod traffic, use network policies.

若要使用網路原則,必須在建立 AKS 叢集時啟用該功能。To use network policy, the feature must be enabled when you create an AKS cluster. 您無法在現有的 AKS 叢集上啟用網路原則。You can't enable network policy on an existing AKS cluster. 事先規劃以確保在叢集上啟用網路原則,並可以視需要使用它們。Plan ahead to make sure that you enable network policy on clusters and can use them as needed. 網路原則應該只用於以 Linux 為基礎的節點和 AKS 中的 pod。Network policy should only be used for Linux-based nodes and pods in AKS.

使用 YAML 資訊清單將網路原則建立為 Kubernetes 資源。A network policy is created as a Kubernetes resource using a YAML manifest. 原則會套用至已定義的 Pod,然後輸入或輸出規則可定義流量的流動方式。The policies are applied to defined pods, then ingress or egress rules define how the traffic can flow. 下列範例將網路原則套用至已套用 app: backend 標籤的 Pod。The following example applies a network policy to pods with the app: backend label applied to them. 然後,輸入規則僅允許來自具有 app: frontend 標籤的 Pod 的流量:The ingress rule then only allows traffic from pods with the app: frontend label:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: backend-policy
spec:
  podSelector:
    matchLabels:
      app: backend
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend

若要開始使用原則,請參閱保護使用網路原則的 Azure Kubernetes Service (AKS) 中的 pod 之間的流量To get started with policies, see Secure traffic between pods using network policies in Azure Kubernetes Service (AKS).

透過防禦主機安全地連線到節點Securely connect to nodes through a bastion host

最佳做法指引 - 不要將遠端連線公開給 AKS 節點。Best practice guidance - Don't expose remote connectivity to your AKS nodes. 在管理虛擬網路中建立防禦主機或跳躍箱 (jump box)。Create a bastion host, or jump box, in a management virtual network. 您可以使用防禦主機安全地將流量路由到 AKS 叢集,以完成遠端管理工作。Use the bastion host to securely route traffic into your AKS cluster to remote management tasks.

AKS 中的大部分作業都可使用 Azure 管理工具或透過 Kubernetes API 伺服器來完成。Most operations in AKS can be completed using the Azure management tools or through the Kubernetes API server. AKS 節點不會連線至公用網際網路,並僅在私人網路上提供。AKS nodes aren't connected to the public internet, and are only available on a private network. 若要連線至節點並執行維護工作,或針對問題進行疑難排解,請透過防禦主機或跳躍箱 (jump box) 來路由您的連線。To connect to nodes and perform maintenance or troubleshoot issues, route your connections through a bastion host, or jump box. 此主機應位於與 AKS 叢集虛擬網路安全對等互連的個別管理虛擬網路。This host should be in a separate management virtual network that is securely peered to the AKS cluster virtual network.

使用防禦主機或跳躍箱 (jump box) 連線到 AKS 節點

防禦主機的管理網路也應該受到保護。The management network for the bastion host should be secured, too. 使用Azure ExpressRoute or VPN gateway連接到內部部署網路,並使用網路安全性群組控制存取。Use an Azure ExpressRoute or VPN gateway to connect to an on-premises network, and control access using network security groups.

後續步驟Next steps

本文著重於網路連線和安全性。This article focused on network connectivity and security. 如需有關在 Kubernetes 中的網路基本概念的詳細資訊,請參閱網路的 Azure Kubernetes Service (AKS) 中的應用程式概念For more information about network basics in Kubernetes, see Network concepts for applications in Azure Kubernetes Service (AKS)