Jobs

Azure Pipelines | TFS 2018 | TFS 2017

Note

Build and release pipelines are called definitions in Microsoft Team Foundation Server (TFS) 2018 and in older versions. Service connections are called service endpoints in TFS 2018 and in older versions.

You can organize your build or deployment pipeline into jobs. Every build or deployment pipeline has at least one job.

Note

You must install TFS 2018.2 to use jobs in build processes. In TFS 2018 RTM you can use jobs in release deployment processes.

You can organize your deployment process into jobs. Every deployment process has at least one job.

A job is a series of tasks that run sequentially on the same target. At design time in your job you specify a series of tasks that you want to run on a common target. At run time (when either the build or release pipeline is triggered), each job is dispatched as one or more jobs to its target.

Note

You must install Update 2 to use jobs in TFS 2017, and they are available only in release deployment processes. Jobs in build pipelines are available in Azure Pipelines, TFS 2018.2, and newer versions.

In a build pipeline, the most common target is an agent. The other kind of target is the Azure Pipelines server.

In a build pipeline, the most common target is an agent. The other kind of target is the server (your TFS instance).

In a deployment pipeline, the target can be either an agent, a deployment group, or the server.

When the target is an agent, the tasks are run on the computer that hosts the agent.

The full syntax to specify an agent job is:


jobs:
- job: string
  timeoutInMinutes: number
  cancelTimeoutInMinutes: number
  strategy:
    maxParallel: number
    # note: `parallel` and `matrix` are mutually exclusive
    # you may specify one or the other; including both is an error
    parallel: number
    matrix: { string: { string: string } }
  pool:
    name: string
    demands: string | [ string ]

  container: string

  steps:
    - script: echo Hello world

The above syntax is necessary only if you want to define multiple jobs or change the properties for a job. You can skip the job syntax if you need only a single job with the standard options. For example, the following YAML file runs a single job to print Hello world.

steps:
- script: echo Hello world

If you want to specify just the pool, you can do that and skip the other properties. For example:

jobs:
- job: Run this job on a Linux agent
  pool: Default # the default self-hosted agent pool in your organization
  steps:
    ...

It's possible to re-use some or all of a pipeline through templates.

YAML is not yet supported in TFS.

Demands

Use demands to specify what capabilities an agent must have to run jobs from your job.

pool:
  name: myPrivateAgents
  demands: agent.os -equals Windows_NT
steps:
- script: echo hello world

Or multiple demands:

pool:
  name: myPrivateAgents
  demands:
  - agent.os -equals Darwin
  - anotherCapability -equals somethingElse
steps:
- script: echo hello world

YAML is not yet supported in TFS.

Learn more about agent capabilities.

Container image

If you are using YAML, you can specify a Docker container to use for your agent job.

For example, to run a script in a container:

resources:
  containers:
  - container: dev1
    image: ubuntu:16.04
jobs:
- job: job_container
  pool:
    name: default
  container: dev1
  steps:
  - script: printenv

To run your job as four jobs on four different containers:

resources:
  containers:
  - container: dev1
    image: ubuntu:14.04
  - container: dev2
    image: private:ubuntu14
    endpoint: privatedockerhub
  - container: dev3
    image: ubuntu:16.04
    options: --cpu-count 4
  - container: dev4
    image: ubuntu:16.04
    options: --hostname container-test --ip 192.168.0.1
    localImage: true
    env:
      envVariable1: envValue1
      envVariable2: envValue2
jobs:
- job: job_container
  container: $[variables['runtimeContainer']]
  pool:
    name: default
    matrix:
      container_1:
        runtimeContainer: dev1
      container_2:
        runtimeContainer: dev2
      container_3:
        runtimeContainer: dev3
      container_4:
        runtimeContainer: dev4
  steps:
  - script: printenv

YAML is not yet supported in TFS.

Timeouts

To avoid taking up resources when your job is hung or waiting too long, it's a good idea to set a limit on how long your job is allowed to run. Use the job timeout setting to specify the limit in minutes for running the job. Setting the value to zero means that the job can run:

  • Forever on self-hosted agents
  • For 360 minutes (6 hours) on Microsoft-hosted agents with a public project and public repository
  • For 30 minutes on Microsoft-hosted agents with a private project or private repository

The timeoutInMinutes allows setting a limit on the job execution time. When not specified, the default is 60 minutes. When 0 is specified, the maximum limit is used (described above).

The cancelTimeoutInMinutes allows setting a limit on how long the job is allowed to finish when it is cancelled. When not specified, the default is 5 minutes. Set this value to allow sufficient time for steps to complete in cases where you've specified that a task should run even if a previous task has failed, even if the build was cancelled, or with custom conditions that allow a task to always run after cancellation.

jobs:
- job: Test
  timeoutInMinutes: 10
  cancelTimeoutInMinutes: 2

YAML is not yet supported in TFS.

Jobs targeting Microsoft-hosted agents have additional restrictions on how long they may run.

You can also set the timeout for each task individually - see task control options.

Multi-configuration

From a single job you can run multiple jobs and multiple agents in parallel. Some examples include:

  • Multi-configuration builds: An agent job can be used in a build pipeline to build multiple configurations in parallel. For example, you could build a Visual C++ app for both debug and release configurations on both x86 and x64 platforms. To learn more, see Visual Studio Build - multiple configurations for multiple platforms.

  • Multi-configuration deployments: An agent job can be used in a stage of a release pipeline to run multiple deployment jobs in parallel, for example, to different geographic regions.

  • Multi-configuration testing: An agent job can be used in a build pipeline or in a stage of a release pipeline to run a set of tests in parallel - once for each test configuration.

The matrix strategy enables a job to be dispatched multiple times, with different variable sets. The maxParallel tag restricts the amount of parallelism. The following job will be dispatched three times with the values of Location and Browser set as specified. However, only two jobs will run at the same time.

jobs:
- job: Test
  strategy:
    maxParallel: 2
    matrix: 
      US_IE:
        Location: US
        Browser: IE
      US_Chrome:
        Location: US
        Browser: Chrome
      Europe_Chrome:
        Location: Europe
        Browser: Chrome

YAML is not yet supported in TFS.

Slicing

An agent job can be used to run a suite of tests in parallel. For example, you can run a large suite of 1000 tests on a single agent. Or, you can use two agents and run 500 tests on each one in parallel. To leverage slicing, the tasks in the job should be smart enough to understand the slice they belong to. The Visual Studio Test task is one such task that supports test slicing. If you have installed multiple agents, you can specify how the Visual Studio Test task will run in parallel on these agents.

The parallel strategy enables a job to be duplicated many times. The maxParallel tag restricts the amount of parallelism. Variables System.JobPositionInPhase and System.TotalJobsInPhase are added to each job. The variables can then be used within your scripts to divide work among the jobs. See Parallel and multiple execution using agent jobs.

The following job will be dispatched 5 times with the values of System.JobPositionInPhase and System.TotalJobsInPhase set appropriately. However, only two jobs will run at the same time.

jobs:
- job: Test
  strategy:
    parallel: 5
    maxParallel: 2

YAML is not yet supported in TFS.

Job variables

If you are using YAML, variables can be specified on the job. The variables can be passed to task inputs using the macro syntax $(variableName), or accessed within a script using the stage variable.

Here is an example of defining variables in a job and using them within tasks.

variables:
  mySimpleVar: simple var value
  "my.dotted.var": dotted var value
  "my var with spaces": var with spaces value

steps:
- script: echo Input macro = $(mySimpleVar). Env var = %MYSIMPLEVAR%
  condition: eq(variables['agent.os'], 'Windows_NT')
- script: echo Input macro = $(mySimpleVar). Env var = $MYSIMPLEVAR
  condition: in(variables['agent.os'], 'Darwin', 'Linux')
- bash: echo Input macro = $(my.dotted.var). Env var = $MY_DOTTED_VAR
- powershell: Write-Host "Input macro = $(my var with spaces). Env var = $env:MY_VAR_WITH_SPACES"

YAML is not yet supported in TFS.

Artifact download

# test and upload my code as an artifact named WebSite
jobs:
- job: Build
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - script: npm test
  - task: PublishBuildArtifacts@1
    inputs:
      pathtoPublish: '$(System.DefaultWorkingDirectory)'
      artifactName: WebSite

# download the artifact and deploy it only if the build job succeeded
- job: Deploy
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - checkout: none #skip checking out the default repository resource
  - task: DownloadBuildArtifacts@0
    displayName: 'Download Build Artifacts'
    inputs:
      artifactName: WebSite
      downloadPath: $(System.DefaultWorkingDirectory)

  dependsOn: Build
  condition: succeeded()

Access to OAuth token

You can allow tasks running in this job to access current Azure Pipelines or TFS OAuth security token. The token can be use to authenticate to the Azure Pipelines REST API.

OAuth token is always made available to the scripts that run through YAML. Here is an example:

steps:
- powershell: |
    $url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=4.1-preview"
    Write-Host "URL: $url"
    $pipeline = Invoke-RestMethod -Uri $url -Headers @{
      Authorization = "Bearer $env:TOKEN"
    }
    Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
  env:
    TOKEN: $(system.accesstoken)

YAML is not yet supported in TFS.