Azure Machine Learning Curated Environments
This article lists the curated environments with latest framework versions in Azure Machine Learning. Curated environments are provided by Azure Machine Learning and are available in your workspace by default. They are backed by cached Docker images that use the latest version of the Azure Machine Learning SDK, reducing the run preparation cost and allowing for faster deployment time. Use these environments to quickly get started with various machine learning frameworks.
Note
Use the Python SDK, CLI, or Azure Machine Learning studio to get the full list of environments and their dependencies. For more information, see the environments article.
Training curated environments
PyTorch
Name: AzureML-pytorch-1.10-ubuntu18.04-py38-cuda11-gpu Description: An environment for deep learning with PyTorch containing the AzureML Python SDK and other python packages.
The following Dockerfile can be customized for your personal workflows.
FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.1-cudnn8-ubuntu18.04:20211221.v1
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/pytorch-1.10
# Create conda environment
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
python=3.8 \
pip=20.2.4 \
pytorch=1.10.0 \
torchvision=0.11.1 \
torchaudio=0.10.0 \
cudatoolkit=11.1.1 \
nvidia-apex=0.1.0 \
gxx_linux-64 \
-c anaconda -c pytorch -c conda-forge
# Prepend path to AzureML conda environment
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
# Install pip dependencies
RUN pip install 'matplotlib>=3.3,<3.4' \
'psutil>=5.8,<5.9' \
'tqdm>=4.59,<4.63' \
'pandas>=1.3,<1.4' \
'scipy>=1.5,<1.8' \
'numpy>=1.10,<1.22' \
'ipykernel~=6.0' \
'azureml-core==1.37.0.post1' \
'azureml-defaults==1.37.0' \
'azureml-mlflow==1.37.0' \
'azureml-telemetry==1.37.0' \
'tensorboard==2.6.0' \
'tensorflow-gpu==2.6.0' \
'onnxruntime-gpu>=1.7,<1.10' \
'horovod==0.23' \
'future==0.18.2' \
'torch-tb-profiler==0.3.1'
# This is needed for mpi to locate libpython
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
Other available PyTorch environments:
- AzureML-pytorch-1.9-ubuntu18.04-py37-cuda11-gpu
- AzureML-pytorch-1.8-ubuntu18.04-py37-cuda11-gpu
- AzureML-pytorch-1.7-ubuntu18.04-py37-cuda11-gpu
LightGBM
Name: AzureML-lightgbm-3.2-ubuntu18.04-py37-cpu
Description: An environment for machine learning with Scikit-learn, LightGBM, XGBoost, Dask containing the AzureML Python SDK and other packages.
The following Dockerfile can be customized for your personal workflows.
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20211221.v1
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/lightgbm
# Create conda environment
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
python=3.7 pip=20.2.4
# Prepend path to AzureML conda environment
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
# Install pip dependencies
RUN HOROVOD_WITH_TENSORFLOW=1 \
pip install 'matplotlib>=3.3,<3.4' \
'psutil>=5.8,<5.9' \
'tqdm>=4.59,<4.60' \
'pandas>=1.1,<1.2' \
'numpy>=1.10,<1.20' \
'scipy~=1.5.0' \
'scikit-learn~=0.24.1' \
'xgboost~=1.4.0' \
'lightgbm~=3.2.0' \
'dask~=2021.6.0' \
'distributed~=2021.6.0' \
'dask-ml~=1.9.0' \
'adlfs~=0.7.0' \
'ipykernel~=6.0' \
'azureml-core==1.37.0.post1' \
'azureml-defaults==1.37.0' \
'azureml-mlflow==1.37.0' \
'azureml-telemetry==1.37.0'
# This is needed for mpi to locate libpython
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
Sklearn
Name: AzureML-sklearn-0.24-ubuntu18.04-py37-cuda11-gpu
Description: An environment for tasks such as regression, clustering, and classification with Scikit-learn. Contains the AzureML Python SDK and other python packages.
The following Dockerfile can be customized for your personal workflows.
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20211221.v1
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/sklearn-0.24.1
# Create conda environment
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
python=3.7 pip=20.2.4
# Prepend path to AzureML conda environment
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
# Install pip dependencies
RUN pip install 'matplotlib>=3.3,<3.4' \
'psutil>=5.8,<5.9' \
'tqdm>=4.59,<4.60' \
'pandas>=1.1,<1.2' \
'scipy>=1.5,<1.6' \
'numpy>=1.10,<1.20' \
'ipykernel~=6.0' \
'azureml-core==1.37.0.post1' \
'azureml-defaults==1.37.0' \
'azureml-mlflow==1.37.0' \
'azureml-telemetry==1.37.0' \
'scikit-learn==0.24.1'
# This is needed for mpi to locate libpython
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
TensorFlow
Name: AzureML-tensorflow-2.4-ubuntu18.04-py37-cuda11-gpu
Description: An environment for deep learning with TensorFlow containing the AzureML Python SDK and other python packages.
The following Dockerfile can be customized for your personal workflows.
FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.0.3-cudnn8-ubuntu18.04:20211221.v1
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/tensorflow-2.4
# Create conda environment
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
python=3.7 pip=20.2.4
# Prepend path to AzureML conda environment
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
# Install pip dependencies
RUN HOROVOD_WITH_TENSORFLOW=1 \
pip install 'matplotlib>=3.3,<3.4' \
'psutil>=5.8,<5.9' \
'tqdm>=4.59,<4.60' \
'pandas>=1.1,<1.2' \
'scipy>=1.5,<1.6' \
'numpy>=1.10,<1.20' \
'ipykernel~=6.0' \
'azureml-core==1.37.0.post1' \
'azureml-defaults==1.37.0' \
'azureml-mlflow==1.37.0' \
'azureml-telemetry==1.37.0' \
'tensorboard==2.4.0' \
'tensorflow-gpu==2.4.1' \
'tensorflow-datasets==4.3.0' \
'onnxruntime-gpu>=1.7,<1.8' \
'horovod[tensorflow-gpu]==0.21.3'
# This is needed for mpi to locate libpython
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
Automated ML (AutoML)
Azure ML pipeline training workflows that use AutoML automatically selects a curated environment based on the compute type and whether DNN is enabled. AutoML provides the following curated environments:
| Name | Compute Type | DNN enabled |
|---|---|---|
| AzureML-AutoML | CPU | No |
| AzureML-AutoML-DNN | CPU | Yes |
| AzureML-AutoML-GPU | GPU | No |
| AzureML-AutoML-DNN-GPU | GPU | Yes |
For more information on AutoML and Azure ML pipelines, see use automated ML in an Azure Machine Learning pipeline in Python.
Inference curated environments and prebuilt docker images
- All the docker images run as non-root user.
- We recommend using
latesttag for docker images. Prebuilt docker images for inference are published to Microsoft container registry (MCR), to query list of tags available, follow instructions on their GitHub repository.
TensorFlow
| Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment |
|---|---|---|---|---|
| 1.15 | CPU | pandas==0.25.1 numpy=1.20.1 | mcr.microsoft.com/azureml/tensorflow-1.15-ubuntu18.04-py37-cpu-inference:latest |
AzureML-tensorflow-1.15-ubuntu18.04-py37-cpu-inference |
| 2.4 | CPU | numpy>=1.16.0 pandas~=1.1.x | mcr.microsoft.com/azureml/tensorflow-2.4-ubuntu18.04-py37-cpu-inference:latest |
AzureML-tensorflow-2.4-ubuntu18.04-py37-cpu-inference |
| 2.4 | GPU | numpy >= 1.16.0 pandas~=1.1.x CUDA==11.0.3 CuDNN==8.0.5.39 | mcr.microsoft.com/azureml/tensorflow-2.4-ubuntu18.04-py37-cuda11.0.3-gpu-inference:latest |
AzureML-tensorflow-2.4-ubuntu18.04-py37-cuda11.0.3-gpu-inference |
PyTorch
| Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment |
|---|---|---|---|---|
| 1.6 | CPU | numpy==1.20.1 pandas==0.25.1 | mcr.microsoft.com/azureml/pytorch-1.6-ubuntu18.04-py37-cpu-inference:latest |
AzureML-pytorch-1.6-ubuntu18.04-py37-cpu-inference |
| 1.7 | CPU | numpy>=1.16.0 pandas~=1.1.x | mcr.microsoft.com/azureml/pytorch-1.7-ubuntu18.04-py37-cpu-inference:latest |
AzureML-pytorch-1.7-ubuntu18.04-py37-cpu-inference |
SciKit-Learn
| Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment |
|---|---|---|---|---|
| 0.24.1 | CPU | scikit-learn==0.24.1 numpy>=1.16.0 pandas~=1.1.x | mcr.microsoft.com/azureml/sklearn-0.24.1-ubuntu18.04-py37-cpu-inference:latest |
AzureML-sklearn-0.24.1-ubuntu18.04-py37-cpu-inference |
ONNX Runtime
| Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment |
|---|---|---|---|---|
| 1.6 | CPU | numpy>=1.16.0 pandas~=1.1.x | mcr.microsoft.com/azureml/onnxruntime-1.6-ubuntu18.04-py37-cpu-inference:latest |
AzureML-onnxruntime-1.6-ubuntu18.04-py37-cpu-inference |
XGBoost
| Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment |
|---|---|---|---|---|
| 0.9 | CPU | scikit-learn==0.23.2 numpy==1.20.1 pandas==0.25.1 | mcr.microsoft.com/azureml/xgboost-0.9-ubuntu18.04-py37-cpu-inference:latest |
AzureML-xgboost-0.9-ubuntu18.04-py37-cpu-inference |
No framework
| Framework version | CPU/GPU | Pre-installed packages | MCR Path | Curated environment |
|---|---|---|---|---|
| NA | CPU | NA | mcr.microsoft.com/azureml/minimal-ubuntu18.04-py37-cpu-inference:latest |
AzureML-minimal-ubuntu18.04-py37-cpu-inference |
Security
Version updates for supported environments are released every two weeks to address vulnerabilities no older than 30 days.