Jobs

Azure Pipelines | Azure DevOps Server 2019 | TFS 2018 | TFS 2017

Note

In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions, service connections are called service endpoints, stages are called environments, and jobs are called phases.

You can organize your pipeline into jobs. Every pipeline has at least one job. A job is a series of steps that run sequentially as a unit. In other words, a job is the smallest unit of work that can be scheduled to run.

You can organize your build or release pipeline into jobs. Every pipeline has at least one job. A job is a series of steps that run sequentially as a unit. In other words, a job is the smallest unit of work that can be scheduled to run.

Note

You must install TFS 2018.2 to use jobs in build processes. In TFS 2018 RTM you can use jobs in release deployment processes.

You can organize your release pipeline into jobs. Every release pipeline has at least one job. Jobs are not supported in a build pipeline in this version of TFS.

Note

You must install Update 2 to use jobs in a release pipeline in TFS 2017. Jobs in build pipelines are available in Azure Pipelines, TFS 2018.2, and newer versions.

Specifying jobs

In the simplest case, a pipeline has a single job. In that case, you do not have to explicitly use the job keyword. You can directly specify the steps in your YAML file.

pool:
  vmImage: 'ubuntu-16.04'
steps:
- bash: echo "Hello world"

You may want to specify additional properties on that job. In that case, you can use the job keyword.

job: myJob
timeoutInMinutes: 10
pool:
  vmImage: 'ubuntu-16.04'
steps:
- bash: echo "Hello world"

Your pipeline may have multiple jobs. In that case, use the jobs keyword.

jobs:
- job: A
  steps:
  - bash: echo "A"

- job: B
  steps:
  - bash: echo "B"

Your pipeline may have multiple stages, each with multiple jobs. In that case, use the stages keyword.

stages:
- stage: A
  jobs:
  - job: A1
  - job: A2

- stage: B
  jobs:
  - job: B1
  - job: B2

The full syntax to specify a job is:

- job: string  # name of the job, A-Z, a-z, 0-9, and underscore
  displayName: string  # friendly name to display in the UI
  dependsOn: string | [ string ]
  condition: string
  strategy:
    matrix: # matrix strategy
    parallel: # parallel strategy
    maxParallel: number # maximum number of agents to simultaneously run copies of this job on
    # note: `parallel` and `matrix` are mutually exclusive
    # you may specify one or the other; including both is an error
  continueOnError: boolean  # 'true' if future jobs should run even if this job fails; defaults to 'false'
  pool: pool # see pool schema
  workspace:
    clean: outputs | resources | all # what to clean up before the job runs
  container: containerReference # container to run this job inside
  timeoutInMinutes: number # how long to run the job before automatically cancelling
  cancelTimeoutInMinutes: number # how much time to give 'run always even if cancelled tasks' before killing them
  variables: { string: string } | [ variable | variableReference ] 
  steps: [ script | bash | pwsh | powershell | checkout | task | templateReference ]
  services: { string: string | container } # container resources to run as a service container

If the primary intent of your job is to deploy your app (as opposed to build or test your app), then you can use a special type of job called deployment job.

The syntax for a deployment job is:

- deployment: string        # instead of job keyword, use deployment keyword
  pool:
    name: string
    demands: string | [ string ]
  environment: string
  strategy:
    runOnce:
      deploy:
        steps:
        - script: echo Hi!

Although you can add steps for deployment tasks in a job, we recommend that you instead use a deployment job. A deployment job has a few benefits. For example, you can deploy to an environment, which includes benefits such as being able to see the history of what you've deployed.

YAML is not supported in this version of TFS.

Types of jobs

Jobs can be of different types, depending on where they run.

  • Agent pool jobs run on an agent in an agent pool.
  • Server jobs run on the Azure DevOps server.
  • Container jobs run in a container on an agent in an agent pool. See container jobs for more information.
  • Agent pool jobs run on an agent in an agent pool.
  • Server jobs run on the Azure DevOps server.
  • Agent pool jobs run on an agent in the agent pool. These jobs are available in build and release pipelines.
  • Server jobs run on TFS. These jobs are available in build and release pipelines.
  • Deployment group jobs run on machines in a deployment group. These jobs are available only in release pipelines.
  • Agent pool jobs run on an agent in the agent pool. These jobs are only available release pipelines.

Agent pool jobs

These are the most common type of jobs and they run on an agent in an agent pool. Use demands to specify what capabilities an agent must have to run your job.

pool:
  name: myPrivateAgents    # your job runs on an agent in this pool
  demands: agent.os -equals Windows_NT    # the agent must have this capability to run the job
steps:
- script: echo hello world

Or multiple demands:

pool:
  name: myPrivateAgents
  demands:
  - agent.os -equals Darwin
  - anotherCapability -equals somethingElse
steps:
- script: echo hello world

YAML is not yet supported in TFS.

Learn more about agent capabilities.

Server jobs

Tasks in a server job are orchestrated by and executed on the server (Azure Pipelines or TFS). A server job does not require an agent or any target computers. Only a few tasks are supported in a server job at present.

The full syntax to specify a server job is:

jobs:
- job: string
  timeoutInMinutes: number
  cancelTimeoutInMinutes: number
  strategy:
    maxParallel: number
    matrix: { string: { string: string } }

  pool: server

You can also use the simplified syntax:

jobs:
- job: string
  server: true

YAML is not yet supported in TFS.

Dependencies

When you define multiple jobs in a single stage, you can specify dependencies between them.

Note

Each agent can run only one job at a time. To run multiple jobs in parallel you must configure multiple agents. You also need sufficient parallel jobs.

The syntax for defining multiple jobs and their dependencies is:

jobs:
- job: string
  dependsOn: string
  condition: string

Example jobs that build sequentially:

jobs:
- job: Debug
  steps:
  - script: echo hello from the Debug build
- job: Release
  dependsOn: Debug
  steps:
  - script: echo hello from the Release build

Example jobs that build in parallel (no dependencies):

jobs:
- job: Windows
  pool:
    vmImage: 'vs2017-win2016'
  steps:
  - script: echo hello from Windows
- job: macOS
  pool:
    vmImage: 'macOS-10.13'
  steps:
  - script: echo hello from macOS
- job: Linux
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - script: echo hello from Linux

Example of fan out:

jobs:
- job: InitialJob
  steps:
  - script: echo hello from initial job
- job: SubsequentA
  dependsOn: InitialJob
  steps:
  - script: echo hello from subsequent A
- job: SubsequentB
  dependsOn: InitialJob
  steps:
  - script: echo hello from subsequent B

Example of fan in:

jobs:
- job: InitialA
  steps:
  - script: echo hello from initial A
- job: InitialB
  steps:
  - script: echo hello from initial B
- job: Subsequent
  dependsOn:
  - InitialA
  - InitialB
  steps:
  - script: echo hello from subsequent

YAML builds are not yet available on TFS.

Conditions

You can specify the conditions under which each job runs. By default, a job runs if it does not depend on any other job, or if all of the jobs that it depends on have completed and succeeded. You can customize this behavior by forcing a job to run even if a previous job fails or by specifying a custom condition.

Example to run a job based upon the status of running a previous job:

jobs:
- job: A
  steps:
  - script: exit 1

- job: B
  dependsOn: A
  condition: failed()
  steps:
  - script: echo this will run when A fails

- job: C
  dependsOn:
  - A
  - B
  condition: succeeded('B')
  steps:
  - script: echo this will run when B runs and succeeds

Example of using a custom condition:

jobs:
- job: A
  steps:
  - script: echo hello

- job: B
  dependsOn: A
  condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
  steps:
  - script: echo this only runs for master

You can specify that a job run based on the value of an output variable set in a previous job. In this case, you can only use variables set in directly dependent jobs:

jobs:
- job: A
  steps:
  - script: "echo ##vso[task.setvariable variable=skipsubsequent;isOutput=true]false"
    name: printvar

- job: B
  condition: and(succeeded(), ne(dependencies.A.outputs['printvar.skipsubsequent'], 'true'))
  dependsOn: A
  steps:
  - script: echo hello from B

YAML builds are not yet available on TFS.

Timeouts

To avoid taking up resources when your job is hung or waiting too long, it's a good idea to set a limit on how long your job is allowed to run. Use the job timeout setting to specify the limit in minutes for running the job. Setting the value to zero means that the job can run:

  • Forever on self-hosted agents
  • For 360 minutes (6 hours) on Microsoft-hosted agents with a public project and public repository
  • For 60 minutes on Microsoft-hosted agents with a private project or private repository

The timeout period begins when the job starts running. It does not include the time the job is queued or is waiting for an agent.

The timeoutInMinutes allows a limit to be set for the job execution time. When not specified, the default is 60 minutes. When 0 is specified, the maximum limit is used (described above).

The cancelTimeoutInMinutes allows a limit to be set for the job cancel time. When not specified, the default is 5 minutes.

jobs:
- job: Test
  timeoutInMinutes: 10
  cancelTimeoutInMinutes: 2

YAML is not yet supported in TFS.

Jobs targeting Microsoft-hosted agents have additional restrictions on how long they may run.

You can also set the timeout for each task individually - see task control options.

Multi-configuration

From a single job you author, you can run multiple jobs on multiple agents in parallel. Some examples include:

  • Multi-configuration builds: You can build multiple configurations in parallel. For example, you could build a Visual C++ app for both debug and release configurations on both x86 and x64 platforms. To learn more, see Visual Studio Build - multiple configurations for multiple platforms.

  • Multi-configuration deployments: You can run multiple deployments in parallel, for example, to different geographic regions.

  • Multi-configuration testing: You can run test multiple configurations in parallel.

The matrix strategy enables a job to be dispatched multiple times, with different variable sets. The maxParallel tag restricts the amount of parallelism. The following job will be dispatched three times with the values of Location and Browser set as specified. However, only two jobs will run at the same time.

jobs:
- job: Test
  strategy:
    maxParallel: 2
    matrix: 
      US_IE:
        Location: US
        Browser: IE
      US_Chrome:
        Location: US
        Browser: Chrome
      Europe_Chrome:
        Location: Europe
        Browser: Chrome

Note

Matrix configuration names (like US_IE above) must contain only basic Latin alphabet letters (A-Z, a-z), numbers, and underscores (_). They must start with a letter. Also, they must be 100 characters or less.

It's also possible to use output variables to generate a matrix. This can be handy if you need to generate the matrix using a script.

matrix will accept a runtime expression containing a stringified JSON object. That JSON object, when expanded, must match the matrixing syntax. In the example below, we've hard-coded the JSON string, but it could be generated by a scripting language or command-line program.

jobs:
- job: generator
  steps:
  - bash: echo "##vso[task.setVariable variable=legs;isOutput=true]{'a':{'myvar':'A'}, 'b':{'myvar':'B'}}"
    name: mtrx
  # This expands to the matrix
  #   a:
  #     myvar: A
  #   b:
  #     myvar: B
- job: runner
  dependsOn: generator
  strategy:
    matrix: $[ dependencies.generator.outputs['mtrx.legs'] ]
  steps:
  - script: echo $(myvar) # echos A or B depending on which leg is running

YAML is not supported in TFS.

Slicing

An agent job can be used to run a suite of tests in parallel. For example, you can run a large suite of 1000 tests on a single agent. Or, you can use two agents and run 500 tests on each one in parallel. To leverage slicing, the tasks in the job should be smart enough to understand the slice they belong to. The Visual Studio Test task is one such task that supports test slicing. If you have installed multiple agents, you can specify how the Visual Studio Test task will run in parallel on these agents.

The parallel strategy enables a job to be duplicated many times. The maxParallel tag restricts the amount of parallelism. Variables System.JobPositionInPhase and System.TotalJobsInPhase are added to each job. The variables can then be used within your scripts to divide work among the jobs. See Parallel and multiple execution using agent jobs.

The following job will be dispatched 5 times with the values of System.JobPositionInPhase and System.TotalJobsInPhase set appropriately. However, only two jobs will run at the same time.

jobs:
- job: Test
  strategy:
    parallel: 5
    maxParallel: 2

YAML is not yet supported in TFS.

Job variables

If you are using YAML, variables can be specified on the job. The variables can be passed to task inputs using the macro syntax $(variableName), or accessed within a script using the stage variable.

Here's an example of defining variables in a job and using them within tasks.

variables:
  mySimpleVar: simple var value
  "my.dotted.var": dotted var value
  "my var with spaces": var with spaces value

steps:
- script: echo Input macro = $(mySimpleVar). Env var = %MYSIMPLEVAR%
  condition: eq(variables['agent.os'], 'Windows_NT')
- script: echo Input macro = $(mySimpleVar). Env var = $MYSIMPLEVAR
  condition: in(variables['agent.os'], 'Darwin', 'Linux')
- bash: echo Input macro = $(my.dotted.var). Env var = $MY_DOTTED_VAR
- powershell: Write-Host "Input macro = $(my var with spaces). Env var = $env:MY_VAR_WITH_SPACES"

YAML is not yet supported in TFS.

For information about using a condition, see Specify conditions.

Artifact download

# test and upload my code as an artifact named WebSite
jobs:
- job: Build
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - script: npm test
  - task: PublishBuildArtifacts@1
    inputs:
      pathtoPublish: '$(System.DefaultWorkingDirectory)'
      artifactName: WebSite

# download the artifact and deploy it only if the build job succeeded
- job: Deploy
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - checkout: none #skip checking out the default repository resource
  - task: DownloadBuildArtifacts@0
    displayName: 'Download Build Artifacts'
    inputs:
      artifactName: WebSite
      downloadPath: $(System.DefaultWorkingDirectory)

  dependsOn: Build
  condition: succeeded()

YAML is not yet supported in TFS.

For information about using dependsOn and condition, see Specify conditions.

Access to OAuth token

You can allow scripts running in a job to access the current Azure Pipelines or TFS OAuth security token. The token can be use to authenticate to the Azure Pipelines REST API.

The OAuth token is always available to YAML pipelines. It must be explicitly mapped into the task or step using env. Here's an example:

steps:
- powershell: |
    $url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=4.1-preview"
    Write-Host "URL: $url"
    $pipeline = Invoke-RestMethod -Uri $url -Headers @{
      Authorization = "Bearer $env:TOKEN"
    }
    Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
  env:
    TOKEN: $(system.accesstoken)

YAML is not yet supported in TFS.