Edit

Execute Azure Machine Learning pipelines in Azure Data Factory and Synapse Analytics

  • Article

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

Run your Azure Machine Learning pipelines as a step in your Azure Data Factory and Synapse Analytics pipelines. The Machine Learning Execute Pipeline activity enables batch prediction scenarios such as identifying possible loan defaults, determining sentiment, and analyzing customer behavior patterns.

The below video features a six-minute introduction and demonstration of this feature.

Create a Machine Learning Execute Pipeline activity with UI

To use a Machine Learning Execute Pipeline activity in a pipeline, complete the following steps:

  1. Search for Machine Learning in the pipeline Activities pane, and drag a Machine Learning Execute Pipeline activity to the pipeline canvas.

  2. Select the new Machine Learning Execute Pipeline activity on the canvas if it is not already selected, and its Settings tab, to edit its details.

    Shows the UI for a Machine Learning Execute Pipeline activity.

  3. Select an existing or create a new Azure Machine Learning linked service, and provide details of the pipeline and experiment, and any pipeline parameters or data path assignments required for the pipeline.

Syntax

{
    "name": "Machine Learning Execute Pipeline",
    "type": "AzureMLExecutePipeline",
    "linkedServiceName": {
        "referenceName": "AzureMLService",
        "type": "LinkedServiceReference"
    },
    "typeProperties": {
        "mlPipelineId": "machine learning pipeline ID",
        "experimentName": "experimentName",
        "mlPipelineParameters": {
            "mlParameterName": "mlParameterValue"
        }
    }
}

Type properties

Property Description Allowed values Required
name Name of the activity in the pipeline String Yes
type Type of activity is 'AzureMLExecutePipeline' String Yes
linkedServiceName Linked Service to Azure Machine Learning Linked service reference Yes
mlPipelineId ID of the published Azure Machine Learning pipeline String (or expression with resultType of string) Yes
experimentName Run history experiment name of the Machine Learning pipeline run String (or expression with resultType of string) No
mlPipelineParameters Key, Value pairs to be passed to the published Azure Machine Learning pipeline endpoint. Keys must match the names of pipeline parameters defined in the published Machine Learning pipeline Object with key value pairs (or Expression with resultType object) No
mlParentRunId The parent Azure Machine Learning pipeline run ID String (or expression with resultType of string) No
dataPathAssignments Dictionary used for changing datapaths in Azure Machine learning. Enables the switching of datapaths Object with key value pairs No
continueOnStepFailure Whether to continue execution of other steps in the Machine Learning pipeline run if a step fails boolean No

Note

To populate the dropdown items in Machine Learning pipeline name and ID, the user needs to have permission to list ML pipelines. The UI calls AzureMLService APIs directly using the logged in user's credentials.

Related content

See the following articles that explain how to transform data in other ways: