Variables

Note

In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions, service connections are called service endpoints, stages are called environments, and jobs are called phases.

Variables give you a convenient way to get key bits of data into various parts of the pipeline. As the name suggests, the value of a variable may change from run to run or job to job of your pipeline. Almost any place where a pipeline requires a text string or a number, you can use a variable instead of hard-coding a value. The system will replace the variable with its current value during the pipeline's execution.

Some variables are automatically set by the system. As a pipeline author or end user, you cannot change the value of such variables. See the comprehensive lists of systems variables available as build variables and release variables.

In this topic, we discuss user-defined variables. Names of these variables consist of letters, numbers, ., and _ characters.

Set variables in pipeline

In the most common case, you set the variables and use them within the YAML file. This allows you to track changes to the variable in your version control system. Here is an example that shows how to set two variables - configuration and platform - and use them later in steps. To use a variable in a YAML statement, wrap it in $().

# Set variables once
variables:
  configuration: debug
  platform: x64

steps:

# Use them once
- task: MSBuild@1
  inputs:
    solution: solution1.sln
    configuration: $(configuration) # Use the variable
    platform: $(platform)

# Use them again
- task: MSBuild@1
  inputs:
    solution: solution2.sln
    configuration: $(configuration) # Use the variable
    platform: $(platform)

In the YAML file, you can set a variable at various scopes:

  • At the root level, to make it available to all jobs in the pipeline
  • At the job level, to make it available only to a specific job
variables:
  global_variable: value    # this is available to all jobs

jobs:
- job: job1
  pool:
    vmImage: 'ubuntu-16.04'
  variables:
    job_variable1: value1    # this is only available in job1
  steps:
  - bash: echo $(global_variable)
  - bash: echo $(job_variable1)
  - bash: echo $JOB_VARIABLE1 # variables are available in the script environment too

- job: job2
  pool:
    vmImage: 'ubuntu-16.04'
  variables:
    job_variable2: value2    # this is only available in job2
  steps:
  - bash: echo $(global_variable)
  - bash: echo $(job_variable2)
  - bash: echo $GLOBAL_VARIABLE

Notice that variables are also made available to scripts through environment variables. The syntax for using these environment variables depends on the scripting language. Name is upper-cased, . replaced with _, and automatically inserted into the process environment. Here are some examples:

  • Batch script: %VARIABLE_NAME%
  • PowerShell script: $env:VARIABLE_NAME
  • Bash script: $VARIABLE_NAME

YAML is not supported in TFS.

Secrets

You should not set secret variables in your YAML file. Instead, you should set them in the pipeline editor using the web interface. These variables are scoped to the pipeline in which you set them.

To set secrets in the web interface, follow these steps:

  • Navigate to Pipelines page, select the appropriate pipeline, and then select Edit.
  • Locate the Variables for this pipeline.
  • Add or update the variable.
  • Select the Secret lock icon to store the variable in an encrypted manner.
  • Save the pipeline.

Secret variables are encrypted at rest with a 2048-bit RSA key. They're available on the agent for tasks and scripts to use.

Note

You should never pass secrets on the command line or echo them as output. We make an effort to mask secrets from appearing in output, but things like different encoding or splitting across lines can fool the detection logic. Also, since the secrets are available on the agent, be careful about who has access to alter your pipeline.

Unlike a normal variable, they are not automatically decrypted into environment variables for scripts. You can explicitly map them in, though.

The following example shows how to pass a secret variable called mySecret set in the web interface to a script.

steps:

- powershell: |
    # Using an input-macro:
    Write-Host "This works: $(mySecret)"

    # Using the env var directly:
    Write-Host "This does not work: $env:MYSECRET"

    # Using the mapped env var:
    Write-Host "This works: $env:MY_MAPPED_ENV_VAR"    # Recommended
  env:
    MY_MAPPED_ENV_VAR: $(mySecret)

The output from the above script would look like this:

This works: ***
This does not work:
This works: ***

It is recommended that you use the script's environment in order to pass secrets to the script. Operating systems often log commands for the processes that they run, and you would not want the log to include a secret that you passed in as an input.

Important: By default with GitHub repositories, secret variables associated with your pipeline are not made available to pull request builds of forks. See Validate contributions from forks.

YAML is not supported in TFS.

Share variables across pipelines

To share variables across multiple pipelines in your project, you should set them using variable groups under Library using the web interface. For more information, see variable groups.

Set variables in scripts

A script in your pipeline can define a variable so that it can be consumed by one of the subsequent steps in the pipeline. To set a variable from a script, you use a command syntax and print to stdout.

Set a job-scoped variable from a script

To set a variable from a script, you use the task.setvariable logging command. This does not update the environment variables, but it does make the new variable available to downstream steps within the same job.

pool:
  vmImage: 'ubuntu-16.04'

steps:

# Create a variable or a secret variable
- script: |
    echo '##vso[task.setvariable variable=sauce]crushed tomatoes'
    echo '##vso[task.setvariable variable=secret.Sauce;issecret=true]crushed tomatoes with garlic'

# Print the variable
- script: |
    echo my variable is $(sauce)

Tip

You can run a script on a:

Batch script

icon Set the sauce and secret.Sauce variables

@echo ##vso[task.setvariable variable=sauce]crushed tomatoes
@echo ##vso[task.setvariable variable=secret.Sauce;issecret=true]crushed tomatoes with garlic

icon Read the variables

Arguments

"$(sauce)" "$(secret.Sauce)"

Script

@echo off
set sauceArgument=%~1
set secretSauceArgument=%~2
@echo No problem reading %sauceArgument% or %SAUCE%
@echo But I cannot read %SECRET_SAUCE%
@echo But I can read %secretSauceArgument% (but the log is redacted so I do not spoil
     the secret)

Console output from reading the variables:

No problem reading crushed tomatoes or crushed tomatoes
But I cannot read 
But I can read ******** (but the log is redacted so I do not spoil the secret)

Set a multi-job output variable

If you want to make a variable available to future jobs, you must mark it as an output variable using isOutput=true. Then you can map it into future jobs using $[] syntax and including the step name which set the variable.

jobs:

# Set an output variable from job A
- job: A
  pool:
    vmImage: 'vs2017-win2016'
  steps:
  - powershell: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the value"
    name: setvarStep
  - script: echo $(setvarStep.myOutputVar)
    name: echovar

# Map the variable into job B
- job: B
  dependsOn: A
  pool:
    vmImage: 'ubuntu-16.04'
  variables:
    myVarFromJobA: $[ dependencies.A.outputs['setvarStep.myOutputVar'] ]  # map in the variable
  steps:
  - script: echo $(myVarFromJobA)
    name: echovar

If you're setting a variable from a matrix or slice, then to reference the variable, you have to include the name of the job as well as the step when you access it from a downstream job.

jobs:

# Set an output variable from a job with a matrix
- job: A
  pool:
    vmImage: 'ubuntu-16.04'
  strategy:
    maxParallel: 2
    matrix:
      debugJob:
        configuration: debug
        platform: x64
      releaseJob:
        configuration: release
        platform: x64
  steps:
  - script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the $(configuration) value"
    name: setvarStep
  - script: echo $(setvarStep.myOutputVar)
    name: echovar

# Map the variable from the debug job
- job: B
  dependsOn: A
  pool:
    vmImage: 'ubuntu-16.04'
  variables:
    myVarFromJobADebug: $[ dependencies.A.outputs['debugJob.setvarStep.myOutputVar'] ]
  steps:
  - script: echo $(myVarFromJobADebug)
    name: echovar
jobs:

# Set an output variable from a job with slicing
- job: A
  pool:
    vmImage: 'ubuntu-16.04'
    parallel: 2 # Two slices
  steps:
  - script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the slice $(system.jobPositionInPhase) value"
    name: setvarStep
  - script: echo $(setvarStep.myOutputVar)
    name: echovar

# Map the variable from the job for the first slice
- job: B
  dependsOn: A
  pool:
    vmImage: 'ubuntu-16.04'
  variables:
    myVarFromJobsA1: $[ dependencies.A.outputs['job1.setvarStep.myOutputVar'] ]
  steps:
  - script: "echo $(myVarFromJobsA1)"
    name: echovar

YAML is not supported in TFS.

Set variables using expressions

You can set a variable using an expression. We already encountered one case of this to set a variable to the output of another from a previous job.

- job: B
  dependsOn: A
  variables:
    myVarFromJobsA1: $[ dependencies.A.outputs['job1.setvarStep.myOutputVar'] ]

You can use any of the supported expressions for setting a variable. Here is an example of setting a variable to act as a counter that starts at 100, gets incremented by 1 for every run, and gets reset to 100 every day.

jobs:
- job:
  variables:
    a: $[counter(format('{0:yyyyMMdd}', pipeline.startTime), 100)]
  steps:
    - bash: echo $(a)

For more information about counters and other expressions, see expressions.

YAML is not supported in TFS.

Allow at queue time

You can choose which variables are allowed to be set at queue time and which are fixed by the pipeline author. If a variable appears in the variables block of a YAML file, it is fixed and cannot be overridden at queue time. To allow a variable to be set at queue time, make sure it doesn't appear in the variables block of a pipeline or job. You can set a default value in the editor, and that value can be overridden by the person queuing the pipeline.

YAML is not supported in TFS.

Expansion of variables

When you set a variable with the same name in multiple scopes, the following precedence is used (highest precedence first).

  1. Variable set at queue time
  2. Job level variable set in the YAML file
  3. Pipeline level variable set in the YAML file
  4. Pipeline variable set in the web editor

In the following example, the same variable a is set at the pipeline level and job level in YAML file. It is also set in a variable group G and as a variable in the pipeline using the web editor.

variables:
  a: 'pipeline yaml'

jobs:
- job: A
  variables:
  - group: G
  - name: a
    value: 'job yaml'
  steps:
    - bash: echo $(a)        # This will be 'job yaml'

Important: We have a known issue in the system that does not use the variable set at queue time with the highest precedence. We will be addressing this issue shortly. To workaround this issue, make sure that you only set the variable in the web editor and not in the YAML file.

Variables are expanded once when the run is started, and again, at the beginning of each step. Here is an example to demonstrate this:

jobs:
- job: A
  variables:
  - a: 10
  steps:
    - bash: |
        echo $(a)            # This will be 10
        echo '##vso[task.setvariable variable=a]20'
        echo $(a)            # This will also be 10, since the expansion of $(a) happens before the step
    - bash: echo $(a)        # This will be 20, since the variables are expanded just before the step

There are two steps in the above example, and the expansion of $(a) happens once at the beginning of the run, and once at the beginning of each of the two steps.

YAML is not supported in TFS.