您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

使用 Azure Kubernetes 服务和 Terraform 创建 Kubernetes 群集Create a Kubernetes cluster with Azure Kubernetes Service and Terraform

Azure Kubernetes 服务 (AKS) 管理托管的 Kubernetes 环境,使用户无需具备容器业务流程专业知识即可快速、轻松地部署和管理容器化的应用程序。Azure Kubernetes Service (AKS) manages your hosted Kubernetes environment, making it quick and easy to deploy and manage containerized applications without container orchestration expertise. 它还通过按需预配、升级和缩放资源,消除了正在进行的操作和维护的负担,而无需使应用程序脱机。It also eliminates the burden of ongoing operations and maintenance by provisioning, upgrading, and scaling resources on demand, without taking your applications offline.

本教程介绍如何使用 Terraform 和 AKS 来执行创建 Kubernetes 群集的任务:In this tutorial, you learn how to perform the following tasks in creating a Kubernetes cluster using Terraform and AKS:

  • 使用 HCL(HashiCorp 语言)定义 Kubernetes 群集Use HCL (HashiCorp Language) to define a Kubernetes cluster
  • 使用 Terraform 和 AKS 创建 Kubernetes 群集Use Terraform and AKS to create a Kubernetes cluster
  • 使用 kubectl 工具测试 Kubernetes 群集的可用性Use the kubectl tool to test the availability of a Kubernetes cluster

先决条件Prerequisites

创建目录结构Create the directory structure

首先,创建包含 Terraform 配置文件的目录用于练习。The first step is to create the directory that holds your Terraform configuration files for the exercise.

  1. 浏览到 Azure 门户Browse to the Azure portal.

  2. 打开 Azure Cloud ShellOpen Azure Cloud Shell. 如果事先未选择环境,请选择“Bash”作为环境。 If you didn't select an environment previously, select Bash as your environment.

    Cloud Shell 提示符

  3. 切换到 clouddrive 目录。Change directories to the clouddrive directory.

    cd clouddrive
    
  4. 创建名为 terraform-aks-k8s 的目录。Create a directory named terraform-aks-k8s.

    mkdir terraform-aks-k8s
    
  5. 将目录切换到新目录:Change directories to the new directory:

    cd terraform-aks-k8s
    

声明 Azure 提供程序Declare the Azure provider

创建声明 Azure 提供程序的 Terraform 配置文件。Create the Terraform configuration file that declares the Azure provider.

  1. 在 Cloud Shell 中,创建名为 main.tf 的文件。In Cloud Shell, create a file named main.tf.

    vi main.tf
    
  2. 按 I 键进入插入模式。Enter insert mode by selecting the I key.

  3. 在编辑器中粘贴以下代码:Paste the following code into the editor:

    provider "azurerm" {
        version = "~>1.5"
    }
    
    terraform {
        backend "azurerm" {}
    }
    
  4. Esc 键退出插入模式。Exit insert mode by selecting the Esc key.

  5. 保存文件,然后输入以下命令退出 vi 编辑器:Save the file and exit the vi editor by entering the following command:

    :wq
    

定义 Kubernetes 群集Define a Kubernetes cluster

创建 Terraform 配置文件,用于声明 Kubernetes 群集的资源。Create the Terraform configuration file that declares the resources for the Kubernetes cluster.

  1. 在 Cloud Shell 中,创建名为 k8s.tf 的文件。In Cloud Shell, create a file named k8s.tf.

    vi k8s.tf
    
  2. 按 I 键进入插入模式。Enter insert mode by selecting the I key.

  3. 在编辑器中粘贴以下代码:Paste the following code into the editor:

    resource "azurerm_resource_group" "k8s" {
        name     = "${var.resource_group_name}"
        location = "${var.location}"
    }
    
    resource "random_id" "log_analytics_workspace_name_suffix" {
        byte_length = 8
    }
    
    resource "azurerm_log_analytics_workspace" "test" {
        # The WorkSpace name has to be unique across the whole of azure, not just the current subscription/tenant.
        name                = "${var.log_analytics_workspace_name}-${random_id.log_analytics_workspace_name_suffix.dec}"
        location            = "${var.log_analytics_workspace_location}"
        resource_group_name = "${azurerm_resource_group.k8s.name}"
        sku                 = "${var.log_analytics_workspace_sku}"
    }
    
    resource "azurerm_log_analytics_solution" "test" {
        solution_name         = "ContainerInsights"
        location              = "${azurerm_log_analytics_workspace.test.location}"
        resource_group_name   = "${azurerm_resource_group.k8s.name}"
        workspace_resource_id = "${azurerm_log_analytics_workspace.test.id}"
        workspace_name        = "${azurerm_log_analytics_workspace.test.name}"
    
        plan {
            publisher = "Microsoft"
            product   = "OMSGallery/ContainerInsights"
        }
    }
    
    resource "azurerm_kubernetes_cluster" "k8s" {
        name                = "${var.cluster_name}"
        location            = "${azurerm_resource_group.k8s.location}"
        resource_group_name = "${azurerm_resource_group.k8s.name}"
        dns_prefix          = "${var.dns_prefix}"
    
        linux_profile {
            admin_username = "ubuntu"
    
            ssh_key {
                key_data = "${file("${var.ssh_public_key}")}"
            }
        }
    
        agent_pool_profile {
            name            = "agentpool"
            count           = "${var.agent_count}"
            vm_size         = "Standard_DS1_v2"
            os_type         = "Linux"
            os_disk_size_gb = 30
        }
    
        service_principal {
            client_id     = "${var.client_id}"
            client_secret = "${var.client_secret}"
        }
    
        addon_profile {
            oms_agent {
            enabled                    = true
            log_analytics_workspace_id = "${azurerm_log_analytics_workspace.test.id}"
            }
        }
    
        tags = {
            Environment = "Development"
        }
    }
    

    上面的代码设置群集的名称、位置和 resource_group_name。The preceding code sets the name of the cluster, location, and the resource_group_name. 此外,设置了 dns_prefix 值 - 构成了用于访问群集的完全限定域名 (FQDN) 的一部分。In addition, the dns_prefix value - that forms part of the fully qualified domain name (FQDN) used to access the cluster - is set.

    使用 linux_profile 记录可以配置用于通过 SSH 登录到工作节点的设置。The linux_profile record allows you to configure the settings that enable signing into the worker nodes using SSH.

    使用 AKS 时,只需支付工作节点的费用。With AKS, you pay only for the worker nodes. agent_pool_profile 记录配置这些工作节点的详细信息。The agent_pool_profile record configures the details for these worker nodes. agent_pool_profile 记录包含要创建的工作节点数,以及工作节点的类型。The agent_pool_profile record includes the number of worker nodes to create and the type of worker nodes. 如果将来需要纵向扩展或缩减群集,请修改此记录中的 count 值。If you need to scale up or scale down the cluster in the future, you modify the count value in this record.

  4. Esc 键退出插入模式。Exit insert mode by selecting the Esc key.

  5. 保存文件,然后输入以下命令退出 vi 编辑器:Save the file and exit the vi editor by entering the following command:

    :wq
    

声明变量Declare the variables

  1. 在 Cloud Shell 中,创建名为 variables.tf 的文件。In Cloud Shell, create a file named variables.tf.

    vi variables.tf
    
  2. 按 I 键进入插入模式。Enter insert mode by selecting the I key.

  3. 在编辑器中粘贴以下代码:Paste the following code into the editor:

    variable "client_id" {}
    variable "client_secret" {}
    
    variable "agent_count" {
        default = 3
    }
    
    variable "ssh_public_key" {
        default = "~/.ssh/id_rsa.pub"
    }
    
    variable "dns_prefix" {
        default = "k8stest"
    }
    
    variable cluster_name {
        default = "k8stest"
    }
    
    variable resource_group_name {
        default = "azure-k8stest"
    }
    
    variable location {
        default = "Central US"
    }
    
    variable log_analytics_workspace_name {
        default = "testLogAnalyticsWorkspaceName"
    }
    
    # refer https://azure.microsoft.com/global-infrastructure/services/?products=monitor for log analytics available regions
    variable log_analytics_workspace_location {
        default = "eastus"
    }
    
    # refer https://azure.microsoft.com/pricing/details/monitor/ for log analytics pricing 
    variable log_analytics_workspace_sku {
        default = "PerGB2018"
    }
    
  4. Esc 键退出插入模式。Exit insert mode by selecting the Esc key.

  5. 保存文件,然后输入以下命令退出 vi 编辑器:Save the file and exit the vi editor by entering the following command:

    :wq
    

创建 Terraform 输出文件Create a Terraform output file

使用 Terraform 输出可以定义当 Terraform 应用计划时要向用户突出显示的、可以使用 terraform output 命令查询的值。Terraform outputs allow you to define values that will be highlighted to the user when Terraform applies a plan, and can be queried using the terraform output command. 在本部分,我们将创建一个输出文件,以便使用 kubectl 访问群集。In this section, you create an output file that allows access to the cluster with kubectl.

  1. 在 Cloud Shell 中,创建名为 output.tf 的文件。In Cloud Shell, create a file named output.tf.

    vi output.tf
    
  2. 按 I 键进入插入模式。Enter insert mode by selecting the I key.

  3. 在编辑器中粘贴以下代码:Paste the following code into the editor:

    output "client_key" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config.0.client_key}"
    }
    
    output "client_certificate" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config.0.client_certificate}"
    }
    
    output "cluster_ca_certificate" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config.0.cluster_ca_certificate}"
    }
    
    output "cluster_username" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config.0.username}"
    }
    
    output "cluster_password" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config.0.password}"
    }
    
    output "kube_config" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config_raw}"
    }
    
    output "host" {
        value = "${azurerm_kubernetes_cluster.k8s.kube_config.0.host}"
    }
    
  4. Esc 键退出插入模式。Exit insert mode by selecting the Esc key.

  5. 保存文件,然后输入以下命令退出 vi 编辑器:Save the file and exit the vi editor by entering the following command:

    :wq
    

将 Azure 存储设置为存储 Terraform 状态Set up Azure storage to store Terraform state

Terraform 在本地通过 terraform.tfstate 文件跟踪状态。Terraform tracks state locally via the terraform.tfstate file. 在单用户环境中,此模式非常合适。This pattern works well in a single-person environment. 但是,在更常见的多用户环境中,需要利用 Azure 存储来跟踪服务器上的状态。However, in a more practical multi-person environment, you need to track state on the server utilizing Azure storage. 在本部分,我们将检索所需的存储帐户信息(帐户名称和帐户密钥),并创建用于存储 Terraform 状态信息的存储容器。In this section, you retrieve the necessary storage account information (account name and account key), and create a storage container into which the Terraform state information will be stored.

  1. 在 Azure 门户的左侧菜单中,选择“所有服务”。 In the Azure portal, select All services in the left menu.

  2. 选择“存储帐户”。 Select Storage accounts.

  3. 在“存储帐户”选项卡上,选择用于存储 Terraform 状态信息的存储帐户名称。 On the Storage accounts tab, select the name of the storage account into which Terraform is to store state. 例如,可以使用首次打开 Cloud Shell 时创建的存储帐户。For example, you can use the storage account created when you opened Cloud Shell the first time. Cloud Shell 创建的存储帐户名称通常以 cs 开头,后接由数字和字母组成的随机字符串。The storage account name created by Cloud Shell typically starts with cs followed by a random string of numbers and letters. 请记住选择的存储帐户名称,因为稍后需要用到。Remember the name of the storage account you select, as it is needed later.

  4. 在存储帐户选项卡上,选择“访问密钥”。 On the storage account tab, select Access keys.

    存储帐户菜单

  5. 记下“密钥 1”密钥值。 Make note of the key1 key value. (选择密钥右侧的图标将值复制到剪贴板。)(Selecting the icon to the right of the key copies the value to the clipboard.)

    存储帐户访问密钥

  6. 在 Cloud Shell 中,在 Azure 存储帐户内创建一个容器(请将 <YourAzureStorageAccountName> 和 <YourAzureStorageAccountAccessKey> 占位符替换为 Azure 存储帐户的相应值)。In Cloud Shell, create a container in your Azure storage account (replace the <YourAzureStorageAccountName> and <YourAzureStorageAccountAccessKey> placeholders with the appropriate values for your Azure storage account).

    az storage container create -n tfstate --account-name <YourAzureStorageAccountName> --account-key <YourAzureStorageAccountKey>
    

创建 Kubernetes 群集Create the Kubernetes cluster

本部分介绍如何使用 terraform init 命令来创建资源,这些资源定义了前面部分中所创建的配置文件。In this section, you see how to use the terraform init command to create the resources defined the configuration files you created in the previous sections.

  1. 在 Cloud Shell 中启动 Terraform(请将 <YourAzureStorageAccountName> 和 <YourAzureStorageAccountAccessKey> 占位符替换为 Azure 存储帐户的相应值)。In Cloud Shell, initialize Terraform (replace the <YourAzureStorageAccountName> and <YourAzureStorageAccountAccessKey> placeholders with the appropriate values for your Azure storage account).

    terraform init -backend-config="storage_account_name=<YourAzureStorageAccountName>" -backend-config="container_name=tfstate" -backend-config="access_key=<YourStorageAccountAccessKey>" -backend-config="key=codelab.microsoft.tfstate" 
    

    terraform init 命令显示成功初始化后端和提供程序插件:The terraform init command displays the success of initializing the backend and provider plugin:

    “Terraform init”结果示例

  2. 导出服务主体凭据。Export your service principal credentials. 将 <your-client-id> 和 <your-client-secret> 占位符分别替换为与服务主体关联的 appIdpassword 值。Replace the <your-client-id> and <your-client-secret> placeholders with the appId and password values associated with your service principal, respectively.

    export TF_VAR_client_id=<your-client-id>
    export TF_VAR_client_secret=<your-client-secret>
    
  3. 运行 terraform plan 命令,以创建定义基础结构元素的 Terraform 计划。Run the terraform plan command to create the Terraform plan that defines the infrastructure elements.

    terraform plan -out out.plan
    

    terraform plan 命令显示运行 terraform apply 命令时要创建的资源:The terraform plan command displays the resources that will be created when you run the terraform apply command:

    “Terraform plan”结果示例

  4. 运行 terraform apply 命令,以应用该计划来创建 Kubernetes 群集。Run the terraform apply command to apply the plan to create the Kubernetes cluster. 创建 Kubernetes 群集的过程可能需要花费几分钟时间,从而导致 Cloud Shell 会话超时。如果 Cloud Shell 会话超时,可以遵循“在 Cloud Shell 超时后进行恢复”部分中的步骤来完成本教程。The process to create a Kubernetes cluster can take several minutes, resulting in the Cloud Shell session timing out. If the Cloud Shell session times out, you can follow the steps in the section "Recover from a Cloud Shell timeout" to enable you to complete the tutorial.

    terraform apply out.plan
    

    terraform apply 命令显示创建配置文件中定义的资源的结果:The terraform apply command displays the results of creating the resources defined in your configuration files:

    “Terraform apply”结果示例

  5. 在 Azure 门户中,在左侧菜单中选择“所有服务”,查看为新 Kubernetes 群集创建的资源。 In the Azure portal, select All services in the left menu to see the resources created for your new Kubernetes cluster.

    Cloud Shell 提示符

在 Cloud Shell 超时后进行恢复Recover from a Cloud Shell timeout

如果 Cloud Shell 会话超时,可执行以下步骤予以恢复:If the Cloud Shell session times out, you can perform the following steps to recover:

  1. 启动 Cloud Shell 会话。Start a Cloud Shell session.

  2. 切换到包含 Terraform 配置文件的目录。Change to the directory containing your Terraform configuration files.

    cd /clouddrive/terraform-aks-k8s
    
  3. 运行以下命令:Run the following command:

    export KUBECONFIG=./azurek8s
    

测试 Kubernetes 群集Test the Kubernetes cluster

可以使用 Kubernetes 工具来验证新建的群集。The Kubernetes tools can be used to verify the newly created cluster.

  1. 从 Terraform 状态中获取 Kubernetes 配置,并将其存储在 kubectl 可以读取的文件中。Get the Kubernetes configuration from the Terraform state and store it in a file that kubectl can read.

    echo "$(terraform output kube_config)" > ./azurek8s
    
  2. 设置环境变量,使 kubectl 拾取正确的配置。Set an environment variable so that kubectl picks up the correct config.

    export KUBECONFIG=./azurek8s
    
  3. 验证群集的运行状况。Verify the health of the cluster.

    kubectl get nodes
    

    应会看到工作节点的详细信息,并且这些节点的状态为“就绪”,如下图所示: You should see the details of your worker nodes, and they should all have a status Ready, as shown in the following image:

    使用 kubectl 工具可以验证 Kubernetes 群集的运行状况

监视运行状况和日志Monitor health and logs

创建 AKS 群集以后,已启用监视功能来捕获群集节点和 Pod 的运行状况指标。When the AKS cluster was created, monitoring was enabled to capture health metrics for both the cluster nodes and pods. Azure 门户提供这些运行状况指标。These health metrics are available in the Azure portal. 有关容器运行状况监视的详细信息,请参阅监视 Azure Kubernetes 服务运行状况For more information on container health monitoring, see Monitor Azure Kubernetes Service health.

后续步骤Next steps

本文已介绍如何使用 Terraform 和 AKS 创建 Kubernetes 群集。In this article, you learned how to use Terraform and AKS to create a Kubernetes cluster. 请参阅以下附加资源,帮助自己详细了解 Azure 上的 Terraform:Here are some additional resources to help you learn more about Terraform on Azure:

Microsoft.com 中的 Terraform 中心Terraform Hub in Microsoft.com
Terraform Azure 提供程序文档Terraform Azure provider documentation
Terraform Azure 提供程序源Terraform Azure provider source
Terraform Azure 模块Terraform Azure modules