Edit

Run pipelines with Anaconda environments

  • Article

Azure DevOps Services

Learn how to set up and use Anaconda with Python in your pipeline. Anaconda is a Python distribution for data science and machine learning.

Get started

Follow these instructions to set up a pipeline for a sample Python app with Anaconda environment.

  1. Sign in to your Azure DevOps organization and navigate to your project.

  2. In your project, navigate to the Pipelines page. Then choose the action to create a new pipeline.

  3. Walk through the steps of the wizard by first selecting GitHub as the location of your source code.

  4. You might be redirected to GitHub to sign in. If so, enter your GitHub credentials.

  5. When the list of repositories appears, select your Anaconda sample repository.

  6. Azure Pipelines will analyze the code in your repository and detect an existing azure-pipelines.yml file.

  7. Select Run.

  8. A new run is started. Wait for the run to finish.

Tip

To make changes to the YAML file as described in this topic, select the pipeline in the Pipelines page, and then Edit the azure-pipelines.yml file.

Add conda to your system path

On hosted agents, conda is left out of PATH by default to keep its Python version from conflicting with other installed versions. The task.prependpath agent command will make it available to all subsequent steps.

- bash: echo "##vso[task.prependpath]$CONDA/bin"
  displayName: Add conda to PATH

Create an environment

From command-line arguments

The conda create command will create an environment with the arguments you pass it.

- bash: conda create --yes --quiet --name myEnvironment
  displayName: Create Anaconda environment

From YAML

You can check in an environment.yml file to your repo that defines the configuration for an Anaconda environment.

- script: conda env create --quiet --file environment.yml
  displayName: Create Anaconda environment

Note

If you are using a self-hosted agent and don't remove the environment at the end, you'll get an error on the next build since the environment already exists. To resolve, use the --force argument: conda env create --quiet --force --file environment.yml.

Note

If you are using self-hosted agents that are sharing storage, and running jobs in parallel using the same Anaconda environments, there may be clashes between those environments. To resolve, use the --name argument and a unique identifier as the argument value, like a concatenation with the $(Build.BuildNumber) build variable.

Install packages from Anaconda

The following YAML installs the scipy package in the conda environment named myEnvironment.

- bash: |
    source activate myEnvironment
    conda install --yes --quiet --name myEnvironment scipy
  displayName: Install Anaconda packages

Run pipeline steps in an Anaconda environment

Note

Each build step runs in its own process. When you activate an Anaconda environment, it will edit PATH and make other changes to its current process. Therefore, an Anaconda environment must be activated separately for each step.

- bash: |
    source activate myEnvironment
    python -m pytest --junitxml=junit/unit-test.xml
  displayName: pytest

- task: PublishTestResults@2
  inputs:
    testResultsFiles: 'junit/*.xml'
  condition: succeededOrFailed()

FAQs

Why am I getting a "Permission denied" error?

On Hosted macOS, the agent user does not have ownership of the directory where Miniconda is installed. For a fix, see the "Hosted macOS" tab under Add conda to your system path.

Why does my build stop responding on a conda create or conda install step?

If you forget to pass --yes, conda will stop and wait for user interaction.

Why is my script on Windows stopping after it activates the environment?

On Windows, activate is a Batch script. You must use the call command to resume running your script after activating. See examples of using call in a pipeline.

How can I run my tests with multiple versions of Python?

See Build Python apps in Azure Pipelines.