PublishedPipeline class

Definition

Represents a Pipeline to be submitted without the Python code which constructed it.

In addition, a PublishedPipeline can be used to resubmit a Pipeline with different PipelineParameter values and inputs.

PublishedPipeline(name, graph_id, description, version, published_pipeline_id, status, endpoint, total_run_steps, workspace, continue_on_step_failure=None, _pipeline_provider=None, **kwargs)
Inheritance
azureml.core._portal.HasWorkspacePortal
azureml.core._portal.HasPipelinePortal
PublishedPipeline

Parameters

name
str

The name of the published pipeline.

graph_id
str

The ID of the graph for this published pipeline.

description
str

The description of the published pipeline.

version
str

The published pipeline version.

published_pipeline_id
str

The ID of the published pipeline.

status
str

The status of the published pipeline ('Active' or 'Disabled').

endpoint
str

The REST endpoint URL to submit runs for this pipeline.

total_run_steps
int

The number of steps in this pipeline.

workspace
Workspace

The workspace of the published pipeline.

continue_on_step_failure
bool

Whether to continue execution of other steps in the PipelineRun if a step fails. The default is false.

_pipeline_provider
_PublishedPipelineProvider

The published pipeline provider.

kwargs
dict

Custom keyword arguments, reserved for future development

Remarks

A PublishedPipeline can be created from either a Pipeline or a PipelineRun.

An example to publish from a Pipeline is as follows:


   from azureml.pipeline.core import Pipeline

   pipeline = Pipeline(workspace=ws, steps=steps)
   published_pipeline = pipeline.publish(name="My_New_Pipeline",
                                         description="My New Pipeline Description",
                                         version="1.0",
                                         continue_on_step_failure=True)

To publish from a PipelineRun use:


   from azureml.pipeline.core import PipelineRun

   pipeline_run = PipelineRun(experiment=Experiment(ws, "Pipeline_experiment"), run_id="run_id")
   published_pipeline = pipeline_run.publish_pipeline(name="My_New_Pipeline",
                                                      description="My New Pipeline Description",
                                                      version="1.0",
                                                      continue_on_step_failure=True)

Note: the continue_on_step_failure parameter specifies whether the execution of steps in the Pipeline will continue if one step fails. The default value is False, meaning when one step fails, the Pipeline execution will stop, canceling any running steps.

Submit a PublishedPipeline using submit(config, tags=None, **kwargs). When submit is called, a PipelineRun is created which in turn creates StepRun objects for each step in the workflow.

An example to submit a PublishedPipeline is as follows:


   from azureml.pipeline.core import PublishedPipeline

   published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
   pipeline_run = experiment.submit(published_pipeline)

There are a number of optional settings that can be specified when submitting a PublishedPipeline. These include:

  • continue_on_step_failure: Whether to continue execution of other steps in the PipelineRun if a step fails, optional. Only steps that have no dependency on the output of the failed step will continue execution. If provided, this parameter setting overrides the setting on the Pipeline.

  • pipeline_parameters: Parameters to pipeline execution, dictionary of {name: value}. See PipelineParameter for more details.

  • parent_run_id: You can supply a run ID to set the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to the same experiment as the pipeline being submitted.

An example to submit a PublishedPipeline using these settings is as follows:


   from azureml.pipeline.core import PublishedPipeline

   published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
   pipeline_run = experiment.submit(published_pipeline,
                                    continue_on_step_failure=True,
                                    pipeline_parameters={"param1": "value1"},
                                    parent_run_id="<run_id>")

All published pipelines have a REST endpoint. With the pipeline endpoint, you can trigger a run of the pipeline from external systems, such as non-Python clients. For information about how to authenticate when calling REST endpoints, see https://aka.ms/pl-restep-auth.

Using the endpoint enables "managed repeatability" in batch scoring and retraining scenarios, for example. For more information, see https://aka.ms/pl-first-pipeline.

Methods

disable()

Set the published pipeline to 'Disabled' and unavailable to run.

enable()

Set the published pipeline to 'Active' and available to run.

get(workspace, id, _workflow_provider=None, _service_endpoint=None)

Get the published pipeline.

get_graph(_workflow_provider=None)

Get the graph of the PublishedPipeline.

get_step_names(_workflow_provider=None)

Get the list of names of steps in the PublishedPipeline.

list(workspace, active_only=True, _service_endpoint=None)

Get all published pipelines in the current workspace.

save(path=None, _workflow_provider=None)

Save the Pipeline YAML to a file.

submit(workspace, experiment_name, pipeline_parameters=None, _workflow_provider=None, _service_endpoint=None, parent_run_id=None, continue_on_step_failure=None)

Submit the published pipeline. This is equivalent to using submit(config, tags=None, **kwargs).

Returns the submitted PipelineRun. Use this object to monitor and view details of the run.

disable()

Set the published pipeline to 'Disabled' and unavailable to run.

disable()

enable()

Set the published pipeline to 'Active' and available to run.

enable()

get(workspace, id, _workflow_provider=None, _service_endpoint=None)

Get the published pipeline.

get(workspace, id, _workflow_provider=None, _service_endpoint=None)

Parameters

workspace
Workspace

The workspace the published pipeline was created in.

id
str

The ID of the published pipeline.

_workflow_provider
_AevaWorkflowProvider object

The workflow provider.

default value: None
_service_endpoint
str

The service endpoint.

default value: None

Returns

A PublishedPipeline object.

Return type

get_graph(_workflow_provider=None)

Get the graph of the PublishedPipeline.

get_graph(_workflow_provider=None)

Parameters

_workflow_provider
_AevaWorkflowProvider

The workflow provider.

default value: None

Returns

The graph.

Return type

get_step_names(_workflow_provider=None)

Get the list of names of steps in the PublishedPipeline.

get_step_names(_workflow_provider=None)

Parameters

_workflow_provider
_AevaWorkflowProvider

The workflow provider.

default value: None

Returns

The list of the names of steps in the PublishedPipeline.

Return type

list(workspace, active_only=True, _service_endpoint=None)

Get all published pipelines in the current workspace.

list(workspace, active_only=True, _service_endpoint=None)

Parameters

workspace
Workspace

The workspace the published pipeline was created in.

active_only
bool

Whether to return only published pipelines which are currently active.

default value: True
_service_endpoint
str

The service endpoint.

default value: None

Returns

A list of PublishedPipeline objects.

Return type

save(path=None, _workflow_provider=None)

Save the Pipeline YAML to a file.

save(path=None, _workflow_provider=None)

Parameters

path
str

The path to save the YAML to. If the path is a directory, the Pipeline YAML file is saved at path/pipeline_name.yml. If path is None, the current directory is used.

default value: None
_workflow_provider
_AevaWorkflowProvider

The workflow provider.

default value: None

Return type

submit(workspace, experiment_name, pipeline_parameters=None, _workflow_provider=None, _service_endpoint=None, parent_run_id=None, continue_on_step_failure=None)

Submit the published pipeline. This is equivalent to using submit(config, tags=None, **kwargs).

Returns the submitted PipelineRun. Use this object to monitor and view details of the run.

submit(workspace, experiment_name, pipeline_parameters=None, _workflow_provider=None, _service_endpoint=None, parent_run_id=None, continue_on_step_failure=None)

Parameters

workspace
Workspace

The workspace to submit the published pipeline on.

experiment_name
str

The name of the experiment to submit to.

pipeline_parameters
dict

A dictionary of parameters to assign new values {param name, param value}. See PipelineParameter for more details.

default value: None
_workflow_provider
_AevaWorkflowProvider

The workflow provider.

default value: None
_service_endpoint
str

The service endpoint.

default value: None
parent_run_id
str

Optional run ID to set for the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to same experiment as this pipeline is being submitted to.

default value: None
continue_on_step_failure
bool

Whether to continue execution of other steps in the PipelineRun if a step fails, optional. If provided, will override the setting on the Pipeline.

default value: None

Returns

The submitted pipeline run.

Return type

Attributes

continue_on_step_failure

Get the value of the continue_on_step_failure setting.

Returns

The value of the continue_on_step_failure setting.

Return type

description

Get the description of the published pipeline.

Returns

The description of the published pipeline.

Return type

str

endpoint

Get the REST endpoint URL for running a published pipeline.

Returns

The REST endpoint URL for running the published pipeline.

Return type

str

graph_id

Get the ID of the graph for this published pipeline.

Returns

The ID of the graph.

Return type

str

id

Get the published pipeline ID.

Returns

The ID of the published pipeline.

Return type

str

name

Get the name of the published pipeline.

Returns

The published pipeline name.

Return type

str

status

Get the status of the published pipeline.

Returns

The status of the published pipeline.

Return type

str

total_run_steps

Get the number of steps in the pipeline.

Returns

The number of steps in the pipeline.

Return type

int

version

Get the version of the published pipeline.

Returns

The version of the published pipeline.

Return type

str