Install the Azure Machine Learning SDK for Python

This article is a guide for different installation options for the SDK.

Prerequisites


Default install

Use azureml-core.

pip install azureml-core

Then install any other packages required for your particular job.

Upgrade install

Tip

We recommend that you always keep azureml-core updated to the latest version.

Upgrade a previous version:

pip install --upgrade azureml-core

Check version

Verify your SDK version:

pip show azureml-core

To see all packages in your environment:

pip list

You can also show the SDK version in Python, but this version does not include the minor version.

import azureml.core
print(azureml.core.VERSION)

To learn more about how to configure your development environment for Azure Machine Learning service, see Configure your development environment.

Other azureml packages

The SDK contains many other optional packages that you can install. These include dependencies that aren't required for all use-cases, so they are not included in the default installation in order to avoid bloating the environment. The following table outlines the packages ,their use-cases and command to install, update & version check.

Additional package  Use-case Install/Upgrade/Show version                
azureml-automl-core Contains core automated machine learning classes for Azure Machine Learning.
This package is used by azureml-train-automl-client and azureml-train-automl-runtime.
pip install azureml-automl-core
pip install --upgrade azureml-automl-core
pip show azureml-automl-core
azureml-accel-models Accelerates deep neural networks on FPGAs with the Azure ML Hardware Accelerated Models Service. pip install azureml-accel-models
pip install --upgrade azureml-accel-models
pip show azureml-accel-models
azureml-train-automl Provides classes for building and running automated machine learning experiments. Also installs common data science packages including pandas, numpy, and scikit-learn.

If you're looking to submit automated ML runs on a remote compute and don't need do any ML locally, we recommend using the thin client, azureml-train-automl-client, package that is part of the azureml-sdk.

See the additional use-case guidance for more information on installation and working with the full automl SDK or its thin client, azureml-train-automl-client.

Similar to the Python standard, one version backwards and one version forward compatibility is supported, but only for the full azureml-train-automl package. For example, if a model is trained with SDK version 1.29.0, then you can inference with SDK versions between 1.28.0 and 1.30.0.
For local conda environment:
pip install azureml-train-automl
pip install --upgrade azureml-train-automl pip install show azureml-train-automl

Thin client for remote compute:
pip install azureml-train-automl-client
pip install --upgrade azureml-train-automl-client
pip install show azureml-train-automl-client
azureml-contrib Installs azureml-contrib-* packages, which include experimental functionality or preview features. pip install azureml-contrib
pip install --upgrade azureml-contrib
pip show azureml-contrib
azureml-datadrift Contains functionality to detect when model training data has drifted from its scoring data. pip install azureml-datadrift
pip install --upgrade azureml-datadrift
pip show azureml-datadrift
azureml-interpret Used for model interpretability, including feature and class importance for blackbox and whitebox models. pip azureml-interpret
pip install --upgrade azureml-interpret
pip show azureml-interpret
azureml-widgets Contains core packages, modules, and classes for Azure Machine Learning. pip install azureml-widgets
pip install --upgrade azureml-widgets
pip show azureml-widgets
azureml-contrib-services Provides functionality for scoring scripts to request raw HTTP access. pip install azureml-contrib-services
pip install --upgrade azureml-contrib-services
pip show azureml-contrib-services
azureml-tensorboard Provides classes and methods for exporting experiment run history and launching TensorBoard for visualizing experiment performance and structure. pip install azureml-tensorboard
pip install --upgrade azureml-tensorboard
pip show azureml-tensorboard
azureml-mlflow Contains functionality integrating Azure Machine Learning with MLFlow. pip install azureml-mlflow
pip install --upgrade azureml-mlflow
pip show azureml-mlflow
azureml-automl-runtime Contains automated machine learning classes for executing runs in Azure Machine Learning. pip install azureml-automl-runtime
pip install --upgrade azureml-automl-runtime
pip show azureml-automl-runtime
azureml-widgets Contains functionality to view the progress of machine learning training runs in Jupyter Notebooks. pip install azureml-widgets
pip install --upgrade azureml-widgets
pip show azureml-widgets
azureml-train-restclients-hyperdrive Contains classes needed to create HyperDriveRuns with azureml-train-core. pip install azureml-train-restclients-hyperdrive
pip install --upgrade azureml-train-restclients-hyperdrive
pip show azureml-train-restclients-hyperdrive
azureml-train-core Contains base estimator classes and the generic estimator class ,estimators used in Deep Neural Network (DNN) training ,estimators used in Scikit-Learn training, modules and classes supporting hyperparameter tuning. pip install azureml-core
pip install --upgrade azureml-core
pip show azureml-core
azureml-train-automl-runtime Contains functionality representing core automated ML and runtime components in Azure Machine Learning. pip install azureml-train-automl-runtime
pip install --upgrade azureml-train-automl-runtime
pip show azureml-train-automl-runtime
azureml-train-automl-client Contains core packages, modules, and classes for Azure Machine Learning. pip install azureml-train-automl-client
pip install --upgrade azureml-train-automl-client
pip show azureml-train-automl-client
azureml-telemetry This package is used to collect telemetry data like log messages, metrics, events, and activity messages. pip install azureml-telemetry
pip install --upgrade azureml-telemetry
pip show azureml-telemetry
azureml-synapse Contains Magic command to manage Synapse session and submit code and SparkMonitor widget to monitor the spark job progress, for both Jupyter and JupyterLab pip install azureml-synapse
pip install --upgrade azureml-synapse
pip show azureml-synapse
azureml-sdk Thos package is used to build and run machine learning workflows upon the Azure Machine Learning service pip install azureml-sdk
pip install --upgrade azureml-sdk
pip show azureml-sdk
azureml-pipeline-steps Contains pre-built steps that can be executed in an Azure Machine Learning Pipeline. pip install azureml-pipeline-steps
pip install --upgrade azureml-pipeline-steps
pip show azureml-pipeline-steps
azureml-pipeline-core Contains core functionality for Azure Machine Learning pipelines, which are configurable machine learning workflows. pip install azureml-pipeline-core
pip install --upgrade azureml-pipeline-core
pip show azureml-pipeline-core
azureml-pipeline This package is used to build, optimize, and manage machine learning workflows pip install azureml-pipeline
pip install --upgrade azureml-pipeline
pip show azureml-pipeline
azureml-opendatasets Contains core functionality for Azure Machine Learning pipelines, which are configurable machine learning workflows. pip install azureml-opendatasets
pip install --upgrade azureml-opendatasets
pip show azureml-opendatasets
azureml-interpret Contains functionality for working with model interpretability in Azure Machine Learning. pip install azureml-interpret
pip install --upgrade azureml-interpret
pip show azureml-interpret
azureml-defaults This package is a metapackage that is used internally by Azure Machine Learning. pip install azureml-defaults
pip install --upgrade azureml-defaults
pip show azureml-defaults
azureml-dataset-runtime The purpose of this package is to coordinate dependencies within AzureML packages. This package is internal, and is not intended to be used directly. pip install azureml-dataset-runtime
pip install --upgrade azureml-dataset-runtime
pip show azureml-dataset-runtime
azureml-datadrift Contains functionality to detect when model training data has drifted from its scoring data. pip install azureml-datadrift
pip install --upgrade azureml-datadrift
pip show azureml-datadrift
azureml-contrib-server This package is local HTTP service used to expose a subset of the functionality provided by the AzureML SDK to VS Tools for AI extensions (VSCode and Visual Studio) pip install azureml-contrib-server
pip install --upgrade azureml-contrib-server
pip show azureml-contrib-server
azureml-contrib-run This package is used to contain the integration code of AzureML with Mlflow. pip install azureml-core
pip install --upgrade azureml-core
pip show azureml-core
azureml-contrib-reinforcementlearning Contains functionality for creating a Windows compute target in Azure Machine Learning. pip install azureml-contrib-reinforcementlearning
pip install --upgrade azureml-contrib-reinforcementlearning
pip show azureml-contrib-reinforcementlearning
azureml-contrib-pipeline-steps Contains modules and classes for specialized Azure Machine Learning Pipeline steps and associated configuration. pip install azureml-contrib-pipeline-steps
pip install --upgrade azureml-contrib-pipeline-steps
pip show azureml-contrib-pipeline-steps
azureml-contrib-notebook Contains extensions for working with Jupyter notebooks in Azure Machine Learning. pip install azureml-contrib-notebook
pip install --upgrade azureml-contrib-notebook
pip show azureml-contrib-notebook
azureml-contrib-gbdt This package contains LightGBM estimator. pip install azureml-contrib-gbdt
pip install --upgrade azureml-contrib-gbdt
pip show azureml-contrib-gbdt
azureml-contrib-functions Contains functionality for packaging Azure Machine Learning models for deployment to Azure Functions. pip install azureml-contrib-functions
pip install --upgrade azureml-contrib-functions
pip show azureml-contrib-functions
azureml-contrib-fairness This package supports the use of fairness assessment dashboards in Azure Machine Learning Studio pip install azureml-contrib-fairness
pip install --upgrade azureml-contrib-fairness
pip show azureml-contrib-fairness
azureml-contrib-dataset Contains specialized functionality for working with Dataset objects in Azure Machine Learning. pip install azureml-contrib-dataset
pip install --upgrade azureml-contrib-dataset
pip show azureml-contrib-dataset
azureml-contrib-automl-pipeline-steps Contains pre-built steps that can be executed in an Azure Machine Learning Pipeline. pip install azureml-contrib-automl-pipeline-steps
pip install --upgrade azureml-contrib-automl-pipeline-steps
pip show azureml-contrib-automl-pipeline-steps
azureml-contrib-automl-dnn-vision This package is only meant to be used by AutoML system-generated scripts. To install in Windows, the “torch” and “torchvision” packages must be installed separately before this package. pip install azureml-contrib-automl-dnn-vision
pip install --upgrade azureml-contrib-automl-dnn-vision
pip show azureml-contrib-automl-dnn-vision
azureml-contrib-automl-dnn-forecasting Azure ML CLI extension common package. Common across azure-cli-ml and azure-cli-ml-preview. pip install azureml-contrib-automl-dnn-forecasting
pip install --upgrade azureml-contrib-automl-dnn-forecasting
pip show azureml-contrib-automl-dnn-forecasting
azureml-contrib-aisc AzureML Contrib for AzureML AI Super Computer compute target. AISCCompute is a managed AI compute infrastructure, which can be attached to a workspace by cluster admin. pip install azureml-contrib-aisc
pip install --upgrade azureml-contrib-aisc
pip show azureml-contrib-aisc
azureml-cli-common Azure ML CLI extension common package. Common across azure-cli-ml and azure-cli-ml-preview. pip install azureml-cli-common
pip install --upgrade azureml-cli-common
pip show azureml-cli-common
azureml-automl-dnn-nlp This package is only meant to be used by AutoML system-generated scripts. pip install azureml-automl-dnn-nlp
pip install --upgrade azureml-automl-dnn-nlp
pip show azureml-automl-dnn-nlp
azureml-accel-models Accelerate deep neural networks on FPGAs with the Azure ML Hardware Accelerated Models Service. pip install azureml-accel-models
pip install --upgrade azureml-accel-models
pip show azureml-accel-models
azureml-inference-server-http This package enable Local Development,CI/CD Integration,Server Routes. pip install azureml-inference-server-http
pip install --upgrade azureml-inference-server-http
pip show azureml-inference-server-http
azure-ml-component This package contains functionality for authoring and managing Azure Machine Learning components authoring and submiting pipelines using components pip install azure-ml-component
pip install --upgrade azure-ml-component
pip show azure-ml-component
azureml-pipeline-wrapper This package contains functionality for authoring and managing Azure Machine Learning modules , authoring and submiting pipelines using modules pip install azureml-pipeline-wrapper
pip install --upgrade azureml-pipeline-wrapper
pip show azureml-pipeline-wrapper
azureml-designer-cv-modules Modules to pre-process and transform images such as to crop, pad or resize. pip install azureml-designer-cv-modules
pip install --upgrade azureml-designer-cv-modules
pip show azureml-designer-cv-modules
azureml-designer-pytorch-modules Modules to train and inference image classification models based on pytorch framework. pip install azureml-designer-pytorch-modules
pip install --upgrade azureml-designer-pytorch-modules
pip show azureml-designer-pytorch-modules
azureml-designer-vowpal-wabbit-modules Modules to train and inference models based on Vowpal Wabbit framework. pip install azureml-designer-vowpal-wabbit-modules
pip install --upgrade azureml-designer-vowpal-wabbit-modules
pip show azureml-designer-vowpal-wabbit-modules
azureml-designer-classic-modules A variety of modules for data processing, model training, inferencing and evaluation. pip install azureml-designer-classic-modules
pip install --upgrade azureml-designer-classic-modules
pip show azureml-designer-classic-modules
azureml-designer-recommender-modules Modules to train and inference recommendation models based on deep neural network. pip install azureml-designer-recommender-modules
pip install --upgrade azureml-designer-recommender-modules
pip show azureml-designer-recommender-modules
azureml-designer-internal Internal functionalities provided for built-in modules. pip install azureml-designer-internal
pip install --upgrade azureml-designer-internal
pip show azureml-designer-internal
azureml-designer-core Core functionalities for data-type definition, data io and frequently-used functions. pip install azureml-designer-core
pip install --upgrade azureml-designer-core
pip show azureml-designer-core
azureml-designer-datatransform-modules Modules to transform dataset, such as by applying math operations, sql queries, clipping outliers or generating a statistics report. pip install azureml-designer-datatransform-modules
pip install --upgrade azureml-designer-datatransform-modules
pip show azureml-designer-datatransform-modules
azureml-designer-dataio-modules Modules to load data into azure machine learning designer and write data to cloud-based storage. pip install azureml-designer-dataio-modules
pip install --upgrade azureml-designer-dataio-modules
pip show azureml-designer-dataio-modules
azureml-designer-serving Provide functionalities to invoke built-in modules in deployment service. pip install azureml-designer-serving
pip install --upgrade azureml-designer-serving
pip show azureml-designer-serving
azureml-contrib-datadrift Contains functionality for data drift detection for various datasets used in machine learning, including training datasets and scoring dataset. pip install azureml-contrib-datadrift
pip install --upgrade azureml-contrib-datadrift
pip show azureml-contrib-datadrift
azureml-contrib-explain-model Contains experimental functionality for the azureml-explain-model package, which offers a variety of services for machine learning model interpretability. pip install azureml-contrib-explain-model
pip install --upgrade azureml-contrib-explain-model
pip show azureml-contrib-explain-model
azureml-contrib-opendatasets This package provides a set of APIs to consume Azure Open Datasets. pip install azureml-contrib-opendatasets
pip install --upgrade azureml-contrib-opendatasets
pip show azureml-contrib-opendatasets
azureml-train-widgets Contains widgets for Jupyter Notebooks to visually track your runs. pip install azureml-train-widgets
pip install --upgrade azureml-train-widgets
pip show azureml-train-widgets

For more details on above packages, see AzureML on pypi.

Additional use-case guidance

If your use-case is described below, note the guidance and any recommended actions.

Use-case Guidance
Using automl  Install the full azureml-train-automl SDK in a new 64-bit Python environment. A new 64-bit environment is required because of a dependency on the LightGBM framework. This package installs and pins specific versions of data science packages for compatibility, which requires a clean environment.

The thin client, azureml-train-automl-client, package doesn't install additional data science packages or require a clean Python environment. We recommend azureml-train-automl-client if you only need to submit automated ML runs to a remote compute, and don't need to submit local runs or download your model locally.

One version backwards and one version forward compatibility is only supported for models trained with the full azureml-train-automl package. For example, if a model is trained with SDK version 1.29.0, then you can inference with SDK versions between 1.28.0 and 1.30.0.
Using Azure Databricks In the Azure Databricks environment, use the library sources detailed in this guide for installing the SDK. Also, see these tips for further information on working with Azure Machine Learning SDK for Python on Azure Databricks.
Using Azure Data Science Virtual Machine Azure Data Science Virtual Machines created after September 27, 2018 come with the Python SDK preinstalled.
Running Azure Machine Learning tutorials or notebooks If you are using an older version of the SDK than the one mentioned in the tutorial or notebook, you should upgrade your SDK. Some functionality in the tutorials and notebooks may require additional Python packages such as matplotlib, scikit-learn, or pandas. Instructions in each tutorial and notebook will show you which packages are required.

Troubleshooting

  • Pip Installation: Dependencies are not guaranteed to be consistent with single-line installation:

    This is a known limitation of pip, as it does not have a functioning dependency resolver when you install as a single line. The first unique dependency is the only one it looks at.

    In the following code azureml-datadrift and azureml-train-automl are both installed using a single-line pip install.

      pip install azureml-datadrift, azureml-train-automl
    

    For this example, let's say azureml-datadrift requires version > 1.0 and azureml-train-automl requires version < 1.2. If the latest version of azureml-datadrift is 1.3, then both packages get upgraded to 1.3, regardless of the azureml-train-automl package requirement for an older version.

    To ensure the appropriate versions are installed for your packages, install using multiple lines like in the following code. Order isn't an issue here, since pip explicitly downgrades as part of the next line call. And so, the appropriate version dependencies are applied.

       pip install azureml-datadrift
       pip install azureml-train-automl 
    
  • Explanation package not guaranteed to be installed when installing the azureml-train-automl-client:

    When running a remote AutoML run with model explanation enabled, you will see an error message "Please install azureml-explain-model package for model explanations." This is a known issue. As a workaround follow one of the steps below:

    1. Install azureml-explain-model locally.
        pip install azureml-explain-model
    
    1. Disable the explainability feature entirely by passing model_explainability=False in the AutoML configuration.
        automl_config = AutoMLConfig(task = 'classification',
                               path = '.',
                               debug_log = 'automated_ml_errors.log',
                               compute_target = compute_target,
                               run_configuration = aml_run_config,
                               featurization = 'auto',
                               model_explainability=False,
                               training_data = prepped_data,
                               label_column_name = 'Survived',
                               **automl_settings)
    
  • Panda errors: Typically seen during AutoML Experiment:

    When manually setting up your environment using pip, you may notice errors (especially from pandas) due to unsupported package versions being installed.

    For example, ModuleNotFoundError: No module named 'pandas.core.internals.managers'; 'pandas.core.internals' is not a package

    In order to prevent such errors, please install the AutoML SDK using the automl_setup.cmd:

    1. Open an Anaconda prompt and clone the GitHub repository for a set of sample notebooks.
    git clone https://github.com/Azure/MachineLearningNotebooks.git
    
    1. cd to the how-to-use-azureml/automated-machine-learning folder where the sample notebooks were extracted and then run:
    automl_setup
    
  • KeyError: 'brand' when running AutoML on local compute or Azure Databricks cluster

    If a new environment was created after June 10, 2020, by using SDK 1.7.0 or earlier, training might fail with this error due to an update in the py-cpuinfo package. (Environments created on or before June 10, 2020, are unaffected, as are experiments run on remote compute because cached training images are used.) To work around this issue, take either of the following two steps:

    • Update the SDK version to 1.8.0 or later (this also downgrades py-cpuinfo to 5.0.0):

      pip install --upgrade azureml-sdk[automl]
      
    • Downgrade the installed version of py-cpuinfo to 5.0.0:

      pip install py-cpuinfo==5.0.0
      
  • Error message: Cannot uninstall 'PyYAML'

    Azure Machine Learning SDK for Python: PyYAML is a distutils installed project. Therefore, we cannot accurately determine which files belong to it if there is a partial uninstall. To continue installing the SDK while ignoring this error, use:

    pip install --upgrade azureml-sdk[notebooks,automl] --ignore-installed PyYAML
    
  • Azure Machine Learning SDK installation failing with an exception: ModuleNotFoundError: No module named 'ruamel' or 'ImportError: No module named ruamel.yaml'

    This issue is getting encountered with the installation of Azure Machine Learning SDK for Python on the latest pip (>20.1.1) in the conda base environment for all released versions of Azure Machine Learning SDK for Python. Refer to the following workarounds:

    • Avoid installing Python SDK on the conda base environment, rather create your conda environment and install SDK on that newly created user environment. The latest pip should work on that new conda environment.

    • For creating images in docker, where you cannot switch away from conda base environment, please pin pip<=20.1.1 in the docker file.

    conda install -c r -y conda python=3.8 pip=20.1.1
    

Next steps

Try these next steps to learn how to use the Azure Machine Learning service SDK for Python:

  1. Read the Azure Machine Learnin Python SDK overview to learn about key classes and design patterns with code samples.
  2. Follow Azure Machine Learning Python Get Started tutorial to begin creating experiments and models.