Use secrets from Azure Key Vault in Azure Pipelines

Azure Pipelines | Azure DevOps Server 2020 | Azure DevOps Server 2019

Note

This tutorial will guide you through working with Azure key vault in your pipeline. Another way of working with secrets is using Secret variables in your Azure Pipeline or referencing secrets in a variable group.

Azure Key Vault helps teams to securely store and manage sensitive information such as API keys, passwords, certificates, etc.

In this tutorial, you will learn about:

  • Creating an Azure Key Vault using the Azure CLI
  • Adding a secret and configuring access to Azure key vault
  • Using secrets in your pipeline

Prerequisites

Create an Azure Key Vault

Azure key vaults can be created and managed through the Azure portal or Azure CLI. We will use Azure CLI in this tutorial

Sign in to the Azure Portal, and then select the Cloud Shell button in the upper-right corner.

  1. If you have more than one Azure subscription associated with your account, use the command below to specify a default subscription. You can use az account list to generate a list of your subscriptions.

    az account set --subscription <your_subscription_name_or_ID>
    
  2. Run the following command to set a default Azure region for your subscription. You can use az account list-locations to generate a list of available regions.

    az configure --defaults location=<your_region>
    

    For example, this command will select the westus2 region:

    az configure --defaults location=westus2
    
  3. Run the following command to create a new resource group.

    az group create --name <your-resource-group>
    
  4. Run the following command to create a new key vault.

    az keyvault create \
      --name <your-key-vault> \
      --resource-group <your-resource-group>
    
  5. Run the following command to create a new secret in your key vault. Secrets are stored as a key value pair. In the example below, Password is the key and mysecretpassword is the value.

    az keyvault secret set \
      --name "Password" \
      --value "mysecretpassword" \
      --vault-name <your-key-vault>
    

Create a project

Sign in to Azure Pipelines. Your browser will then navigate to https://dev.azure.com/your-organization-name and displays your Azure DevOps dashboard.

If you don't have any projects in your organization yet, select Create a project to get started to create a new project. Otherwise, select the New project button in the upper-right corner of the dashboard.

Create a repo

We will use YAML to create our pipeline but first we need to create a new repo.

  1. Sign in to your Azure DevOps organization and navigate to your project.

  2. Go to Repos, and then select Initialize to initialize a new repo with a README.

    Creating the repo

Create a new pipeline

  1. Go to Pipelines, and then select New Pipeline.

  2. Select Azure Repos Git.

    Creating the pipeline

  3. Select the repo you created earlier. It should have the same name as your Azure DevOps project.

  4. Select Starter pipeline.

  5. The default pipeline will include a few scripts that run echo commands. Those are not needed so we can delete them. Your new YAML file will now look like this:

     trigger:
     - main
    
     pool:
       vmImage: 'ubuntu-latest'
    
     steps:
    
  6. Select Show assistant to expand the assistant panel. This panel provides convenient and searchable list of pipeline tasks.

    Showing the pipeline assistant

  7. Search for vault and select the Azure Key Vault task.

    Selecting the Azure Key Vault task

  8. Select and authorize the Azure subscription you used to create your Azure key vault earlier. Select the key vault and select Add to insert the task at the end of the pipeline. This task allows the pipeline to connect to your Azure Key Vault and retrieve secrets to use as pipeline variables.

    Note

    Make secrets available to whole job feature is not currently supported in Azure DevOps Server 2019 and 2020.

    Configuring the Azure Key Vault task

  9. This step is optional. To verify the retrieval and processing of our secret through the pipeline, add the script below to your YAML to write the secret to a text file and publish it for review. This is not recommended and it is for demonstration purposes only.

    - script: echo $(Password) > secret.txt
    
    - publish: secret.txt
    

    Tip

    YAML is very particular about formatting and indentation. Make sure your YAML file is indented properly.

  10. Do not save or run the pipeline yet. It will fail because the pipeline does not have permissions to access the key vault yet. Keep this browser tab open, we will resume once we set up the key vault permissions.

Set up Azure Key Vault access policies

  1. Go to Azure portal.

  2. Use the search bar to search for the key vault you created earlier.

    Searching for Azure Key Vault

  3. Under Settings Select Access policies.

  4. Select Add Access Policy to add a new policy.

  5. For Secret permissions, select Get and List.

  6. Select the option to select a principal and search for yours.

    A security principal is an object that represents a user, group, service, or application that's requesting access to Azure resources. Azure assigns a unique object ID to every security principal. The default naming convention is [Azure DevOps account name]-[Azure DevOps project name]-[subscription ID] so if your account is "https://dev.azure.com/Contoso" and your team project is "AzureKeyVault", your principal would look something like this Contoso-AzureKeyVault-[subscription ID].

    Tip

    You may need to minimize the Azure CLI panel to see the Select button.

  7. Select Add to create the access policy.

  8. Select Save.

Run and review the pipeline

  1. Return to the open pipeline tab where we left off.

  2. Select Save then Save again to commit your changes and trigger the pipeline.

    Note

    You may be asked to allow the pipeline to access Azure resources, if prompted select Allow. You will only have to approve it once.

  3. Select the CmdLine job to view the logs. Note that the actual secret is not part of the logs.

    Reviewing the command line task

  4. Return to pipeline summary and select the published artifact.

    The pipeline summary

  5. Under Job select the secret.txt file to view it.

    Viewing the secret in the artifact

  6. The text file contains our secret: mysecretpassword. This concludes our verification step that we mentioned earlier.

Clean up resources

Follow the steps below to delete the resources you created:

  1. If you created a new organization to host your project, see how to delete your organization, otherwise delete your project).

  2. All Azure resources created during this tutorial are hosted under a single resource group PipelinesKeyVaultResourceGroup. Run the following command to delete the resource group and all of its resources.

    az group delete --name PipelinesKeyVaultResourceGroup
    

Next steps