Azure Kubernetes Service (AKS) 中的應用程式和叢集的安全性概念Security concepts for applications and clusters in Azure Kubernetes Service (AKS)

當您在 Azure Kubernetes Service (AKS) 中執行應用程式工作負載時,若要保護您的客戶資料,叢集的安全性是一項重要考量。To protect your customer data as you run application workloads in Azure Kubernetes Service (AKS), the security of your cluster is a key consideration. Kubernetes 包含 網路原則秘密 等安全性元件。Kubernetes includes security components such as network policies and Secrets. Azure 隨後會新增網路安全性群組和協調的叢集升級等元件。Azure then adds in components such as network security groups and orchestrated cluster upgrades. 這些安全性元件可相互結合,使您的 AKS 叢集執行最新的 OS 安全性更新和 Kubernetes 版本,且具有安全的 Pod 流量和機密認證存取。These security components are combined to keep your AKS cluster running the latest OS security updates and Kubernetes releases, and with secure pod traffic and access to sensitive credentials.

本文將介紹對 AKS 中的應用程式進行保護的核心概念:This article introduces the core concepts that secure your applications in AKS:

主要元件安全性Master security

在 AKS 中,Kubernetes 主要元件包含在 Microsoft 所提供的受控服務中。In AKS, the Kubernetes master components are part of the managed service provided by Microsoft. 每個 AKS 叢集都有自己的單一租使用者專用 Kubernetes 主機,可提供 API 伺服器、排程器等。此主機是由 Microsoft 所管理和維護。Each AKS cluster has its own single-tenanted, dedicated Kubernetes master to provide the API Server, Scheduler, etc. This master is managed and maintained by Microsoft.

根據預設,Kubernetes API 伺服器會使用公用 IP 位址和完整功能變數名稱) (FQDN。By default, the Kubernetes API server uses a public IP address and a fully qualified domain name (FQDN). 您可以使用 授權的 IP 範圍來限制對 API 伺服器端點的存取。You can limit access to the API server endpoint using authorized IP ranges. 您也可以建立完整的 私人 叢集,以將 API 伺服器存取限制為您的虛擬網路。You can also create a fully private cluster to limit API server access to your virtual network.

您可以使用 Kubernetes 角色型存取控制 (Kubernetes RBAC) 和 Azure RBAC 來控制對 API 伺服器的存取。You can control access to the API server using Kubernetes role-based access control (Kubernetes RBAC) and Azure RBAC. 如需詳細資訊,請參閱 Azure AD 與 AKS 的整合For more information, see Azure AD integration with AKS.

節點安全性Node security

AKS 節點是您所管理和維護的 Azure 虛擬機器。AKS nodes are Azure virtual machines that you manage and maintain. Linux 節點會使用 Moby 容器執行時間來執行優化的 Ubuntu 散發套件。Linux nodes run an optimized Ubuntu distribution using the Moby container runtime. Windows Server 節點會執行優化的 Windows Server 2019 版本,同時也會使用 Moby 容器執行時間。Windows Server nodes run an optimized Windows Server 2019 release and also use the Moby container runtime. 當 AKS 叢集建立或相應增加時,節點將會自動以最新的 OS 安全性更新和設定進行部署。When an AKS cluster is created or scaled up, the nodes are automatically deployed with the latest OS security updates and configurations.

Azure 平臺會每晚自動將 OS 安全性修補程式套用至 Linux 節點。The Azure platform automatically applies OS security patches to Linux nodes on a nightly basis. 如果 Linux OS 安全性更新需要主機重新開機,則不會自動執行該重新開機。If a Linux OS security update requires a host reboot, that reboot is not automatically performed. 您可以手動重新開機 Linux 節點,或常見的方法是使用 Kured,這是 Kubernetes 的開放原始碼重新開機背景程式。You can manually reboot the Linux nodes, or a common approach is to use Kured, an open-source reboot daemon for Kubernetes. Kured 會以 DaemonSet 執行,並監視每個節點,查看是否有檔案指示需重新啟動。Kured runs as a DaemonSet and monitors each node for the presence of a file indicating that a reboot is required. 系統會使用相同的 cordon 和 drain 程序作為叢集升級,跨叢集管理作業系統重新啟動。Reboots are managed across the cluster using the same cordon and drain process as a cluster upgrade.

對於 Windows Server 節點,Windows Update 不會自動執行並套用最新的更新。For Windows Server nodes, Windows Update does not automatically run and apply the latest updates. 在 Windows Update 發行週期的定期排程和您自己的驗證程式中,您應該在 AKS 叢集中的 Windows Server 節點集區 (s) 上執行升級。On a regular schedule around the Windows Update release cycle and your own validation process, you should perform an upgrade on the Windows Server node pool(s) in your AKS cluster. 此升級程序會建立節點來執行最新的 Windows Server 映像和修補程式,然後移除較舊的節點。This upgrade process creates nodes that run the latest Windows Server image and patches, then removes the older nodes. 如需此程序的詳細資訊,請參閱在 AKS 中升級節點集區For more information on this process, see Upgrade a node pool in AKS.

節點會部署至私人虛擬網路子網路中,且不會指派公用 IP 位址。Nodes are deployed into a private virtual network subnet, with no public IP addresses assigned. 基於疑難排解和管理用途,依預設會啟用 SSH。For troubleshooting and management purposes, SSH is enabled by default. 此 SSH 存取僅供內部 IP 位址使用。This SSH access is only available using the internal IP address.

若要防止儲存,節點應使用 Azure 受控磁碟。To provide storage, the nodes use Azure Managed Disks. 就大部分的 VM 節點大小而言,這是指採用高效能 SSD 的進階磁碟。For most VM node sizes, these are Premium disks backed by high-performance SSDs. 儲存於受控磁碟上的資料會自動加密,並在 Azure 平台內待用。The data stored on managed disks is automatically encrypted at rest within the Azure platform. 若要提高備援性,這些磁碟也會安全地複寫於 Azure 資料中心內。To improve redundancy, these disks are also securely replicated within the Azure datacenter.

目前,多租用戶如有惡意的使用,AKS 或其他位置中的 Kubernetes 環境就並不完全安全。Kubernetes environments, in AKS or elsewhere, currently aren't completely safe for hostile multi-tenant usage. 其他安全性功能(例如 Pod 安全性原則)或更精細的 Kubernetes 角色型存取控制 (Kubernetes 節點的 RBAC) ,可讓惡意探索變得更困難。Additional security features like Pod Security Policies, or more fine-grained Kubernetes role-based access control (Kubernetes RBAC) for nodes, make exploits more difficult. 不過,在執行惡意的多租用戶工作負載時若要保有真正的安全性,Hypervisor 才是您唯一可信賴的安全性層級。However, for true security when running hostile multi-tenant workloads, a hypervisor is the only level of security that you should trust. Kubernetes 的安全性網域會成為整個叢集,而非個別節點。The security domain for Kubernetes becomes the entire cluster, not an individual node. 對於這些類型的惡意多租用戶工作負載,您應使用實際隔離的叢集。For these types of hostile multi-tenant workloads, you should use physically isolated clusters. 如需有關隔離工作負載方式的詳細資訊,請參閱 AKS 中叢集隔離的最佳作法For more information on ways to isolate workloads, see Best practices for cluster isolation in AKS.

計算隔離Compute isolation

由於合規性或法規需求,某些工作負載可能需要與其他客戶工作負載的高度隔離。Certain workloads may require a high degree of isolation from other customer workloads due to compliance or regulatory requirements. 針對這些工作負載,Azure 會提供 隔離的虛擬機器,可用來做為 AKS 叢集中的代理程式節點。For these workloads, Azure provides isolated virtual machines, which can be used as the agent nodes in an AKS cluster. 這些隔離的虛擬機器會隔離到特定的硬體類型,而且專用於單一客戶。These isolated virtual machines are isolated to a specific hardware type and dedicated to a single customer.

若要搭配 AKS 叢集使用這些隔離的虛擬機器,請在建立 AKS 叢集或新增節點集區時,選取 此處 所列的其中一個隔離的虛擬機器大小作為 節點大小To use these isolated virtual machines with an AKS cluster, select one of the isolated virtual machines sizes listed here as the Node size when creating an AKS cluster or adding a node pool.

叢集升級Cluster upgrades

若要達到安全性與合規性,或是要使用最新功能,可以利用 Azure 提供用來協調 AKS 叢集和元件升級的工具。For security and compliance, or to use the latest features, Azure provides tools to orchestrate the upgrade of an AKS cluster and components. 此升級協調流程包含 Kubernetes 主要元件和代理程式元件。This upgrade orchestration includes both the Kubernetes master and agent components. 您可以針對您的 AKS 叢集,查看 可用的 Kubernetes 版本清單You can view a list of available Kubernetes versions for your AKS cluster. 若要開始執行升級程序,您必須指定其中一個可用版本。To start the upgrade process, you specify one of these available versions. 然後,Azure 會安全地隔離和清空每個 AKS 節點,並執行升級。Azure then safely cordons and drains each AKS node and performs the upgrade.

隔離和清空Cordon and drain

升級過程中,系統會從叢集個別隔離 AKS 節點,因此不會對它們排程新的 pod。During the upgrade process, AKS nodes are individually cordoned from the cluster so new pods aren't scheduled on them. 這些節點接著會清空並升級,如下所示:The nodes are then drained and upgraded as follows:

  • 新節點會部署到節點集區中。A new node is deployed into the node pool. 此節點會執行最新的 OS 映射和修補程式。This node runs the latest OS image and patches.
  • 已識別其中一個現有的節點以進行升級。One of the existing nodes is identified for upgrade. 此節點上的 pod 會正常地終止,並排程在節點集區中的其他節點上。Pods on this node are gracefully terminated and scheduled on the other nodes in the node pool.
  • 此現有節點會從 AKS 叢集中刪除。This existing node is deleted from the AKS cluster.
  • 叢集中的下一個節點會使用相同的程式來隔離和清空,直到所有節點都成功取代為升級程式的一部分。The next node in the cluster is cordoned and drained using the same process until all nodes are successfully replaced as part of the upgrade process.

如需詳細資訊,請參閱升級 AKS 叢集For more information, see Upgrade an AKS cluster.

網路安全性Network security

針對內部部署網路的連線和安全性,您可以將 AKS 叢集部署到現有的 Azure 虛擬網路子網路中。For connectivity and security with on-premises networks, you can deploy your AKS cluster into existing Azure virtual network subnets. 這些虛擬網路可能有連回您內部部署網路的 Azure 站對站 VPN 或 ExpressRoute 連線。These virtual networks may have an Azure Site-to-Site VPN or Express Route connection back to your on-premises network. Kubernetes 輸入控制器可使用私人的內部 IP 位址進行定義,使服務只能透過此內部網路連線來存取。Kubernetes ingress controllers can be defined with private, internal IP addresses so services are only accessible over this internal network connection.

Azure 網路安全性群組Azure network security groups

為了篩選虛擬網路中的流量,Azure 會使用網路安全性群組規則。To filter the flow of traffic in virtual networks, Azure uses network security group rules. 這些規則可定義允許或拒絕存取資源的來源和目的地 IP 範圍、連接埠和通訊協定。These rules define the source and destination IP ranges, ports, and protocols that are allowed or denied access to resources. 系統會建立預設規則,以允許 Kubernetes API 伺服器的 TLS 流量。Default rules are created to allow TLS traffic to the Kubernetes API server. 當您建立具有負載平衡器、連接埠對應或輸入路由的服務時,AKS 將會自動修改網路安全性群組,讓流量以適當方式傳輸。As you create services with load balancers, port mappings, or ingress routes, AKS automatically modifies the network security group for traffic to flow appropriately.

如果您為 AKS 叢集提供自己的子網,而且您想要修改流量流程,請勿修改 AKS 管理的子網層級網路安全性群組。In cases where you provide your own subnet for your AKS cluster and you wish to modify the flow of traffic, do not modify the subnet-level network security group managed by AKS. 您可以建立額外的子網層級網路安全性群組,以修改流量的流量,只要它們不會干擾管理叢集所需的流量,例如負載平衡器存取、與控制平面的通訊,以及輸出。You may create additional subnet-level network security groups to modify the flow of traffic as long as they do not interfere with traffic needed for managing the cluster, such as load balancer access, communication with the control plane, and egress.

Kubernetes 網路原則Kubernetes network policy

為了限制叢集中 pod 之間的網路流量,AKS 提供 Kubernetes 網路原則的支援。To limit network traffic between pods in your cluster, AKS offers support for Kubernetes network policies. 使用網路原則,您可以根據命名空間和標籤選取器,選擇允許或拒絕叢集中的特定網路路徑。With network policies, you can choose to allow or deny specific network paths within the cluster based on namespaces and label selectors.

Kubernetes 秘密Kubernetes Secrets

Kubernetes 祕密 可用來將敏感性資料插入 Pod 中,例如存取認證或金鑰。A Kubernetes Secret is used to inject sensitive data into pods, such as access credentials or keys. 首先,您必須使用 Kubernetes API 建立祕密。You first create a Secret using the Kubernetes API. 您在定義 Pod 或部署時,系統可能會要求特定秘密。When you define your pod or deployment, a specific Secret can be requested. 對於有已排程的 Pod 需要秘密的節點,才會提供秘密,且秘密會儲存在 tmpfs 中,不會寫入至磁碟。Secrets are only provided to nodes that have a scheduled pod that requires it, and the Secret is stored in tmpfs, not written to disk. 當節點上最後一個需要祕密的 Pod 遭刪除時,即會從該節點的 tmpfs 中刪除秘密。When the last pod on a node that requires a Secret is deleted, the Secret is deleted from the node's tmpfs. 祕密儲存在指定的命名空間內,且僅供相同命名空間中的 Pod 存取。Secrets are stored within a given namespace and can only be accessed by pods within the same namespace.

使用祕密可減少在 Pod 或服務 YAML 資訊清單中定義的敏感性資訊。The use of Secrets reduces the sensitive information that is defined in the pod or service YAML manifest. 秘密會以 YAML 資訊清單的形式儲存在 Kubernetes API 伺服器中,供您要求。Instead, you request the Secret stored in Kubernetes API Server as part of your YAML manifest. 此方法僅可供特定 Pod 存取祕密。This approach only provides the specific pod access to the Secret. 請注意:原始密碼資訊清單檔案包含 base64 格式的機密資料 (如需詳細資訊,請參閱 官方檔) 。Please note: the raw secret manifest files contains the secret data in base64 format (see the official documentation for more details). 因此,這個檔案應該視為機密資訊,而且永遠不會認可到原始檔控制。Therefore, this file should be treated as sensitive information, and never committed to source control.

Kubernetes 秘密會儲存在 etcd 中,也就是分散式的索引鍵/值存放區。Kubernetes secrets are stored in etcd, a distributed key-value store. Etcd store 完全受 AKS 管理,且 資料會在 Azure 平臺中待用加密Etcd store is fully managed by AKS and data is encrypted at rest within the Azure platform.

後續步驟Next steps

若要開始保護您的 AKS 叢集,請參閱升級 AKS 叢集To get started with securing your AKS clusters, see Upgrade an AKS cluster.

如需相關的最佳作法,請參閱 AKS 中叢集 安全性和升級的最佳做法 ,以及 AKS 中的 pod 安全性最佳作法For associated best practices, see Best practices for cluster security and upgrades in AKS and Best practices for pod security in AKS.

如需關於 Kubernetes 及 AKS 核心概念的詳細資訊,請參閱下列文章:For additional information on core Kubernetes and AKS concepts, see the following articles: