PublishedPipeline Class

Represents a Pipeline to be submitted without the Python code which constructed it.

In addition, a PublishedPipeline can be used to resubmit a Pipeline with different PipelineParameter values and inputs.

Inheritance
azureml.core._portal.HasPipelinePortal
PublishedPipeline

Constructor

PublishedPipeline(name, graph_id, description, version, published_pipeline_id, status, endpoint, total_run_steps, workspace, continue_on_step_failure=None, _pipeline_provider=None, **kwargs)

Parameters

name
str

The name of the published pipeline.

graph_id
str

The ID of the graph for this published pipeline.

description
str

The description of the published pipeline.

version
str

The published pipeline version.

published_pipeline_id
str

The ID of the published pipeline.

status
str

The status of the published pipeline ('Active' or 'Disabled').

endpoint
str

The REST endpoint URL to submit runs for this pipeline.

total_run_steps
int

The number of steps in this pipeline.

workspace
Workspace

The workspace of the published pipeline.

continue_on_step_failure
bool

Whether to continue execution of other steps in the PipelineRun if a step fails. The default is false.

_pipeline_provider
<xref:_PublishedPipelineProvider>

The published pipeline provider.

kwargs
dict

Custom keyword arguments, reserved for future development

Remarks

A PublishedPipeline can be created from either a Pipeline or a PipelineRun.

An example to publish from a Pipeline is as follows:


   from azureml.pipeline.core import Pipeline

   pipeline = Pipeline(workspace=ws, steps=steps)
   published_pipeline = pipeline.publish(name="My_New_Pipeline",
                                         description="My New Pipeline Description",
                                         version="1.0",
                                         continue_on_step_failure=True)

To publish from a PipelineRun use:


   from azureml.pipeline.core import PipelineRun

   pipeline_run = PipelineRun(experiment=Experiment(ws, "Pipeline_experiment"), run_id="run_id")
   published_pipeline = pipeline_run.publish_pipeline(name="My_New_Pipeline",
                                                      description="My New Pipeline Description",
                                                      version="1.0",
                                                      continue_on_step_failure=True)

Note: the continue_on_step_failure parameter specifies whether the execution of steps in the Pipeline will continue if one step fails. The default value is False, meaning when one step fails, the Pipeline execution will stop, canceling any running steps.

Submit a PublishedPipeline using azureml.core.Experiment.submit. When submit is called, a PipelineRun is created which in turn creates StepRun objects for each step in the workflow.

An example to submit a PublishedPipeline is as follows:


   from azureml.pipeline.core import PublishedPipeline

   published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
   pipeline_run = experiment.submit(published_pipeline)

There are a number of optional settings that can be specified when submitting a PublishedPipeline. These include:

  • continue_on_step_failure: Whether to continue execution of other steps in the PipelineRun if a step fails, optional. Only steps that have no dependency on the output of the failed step will continue execution. If provided, this parameter setting overrides the setting on the Pipeline.

  • pipeline_parameters: Parameters to pipeline execution, dictionary of {name: value}. See PipelineParameter for more details.

  • parent_run_id: You can supply a run ID to set the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to the same experiment as the pipeline being submitted.

An example to submit a PublishedPipeline using these settings is as follows:


   from azureml.pipeline.core import PublishedPipeline

   published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
   pipeline_run = experiment.submit(published_pipeline,
                                    continue_on_step_failure=True,
                                    pipeline_parameters={"param1": "value1"},
                                    parent_run_id="<run_id>")

All published pipelines have a REST endpoint. With the pipeline endpoint, you can trigger a run of the pipeline from external systems, such as non-Python clients. For information about how to authenticate when calling REST endpoints, see https://aka.ms/pl-restep-auth.

Using the endpoint enables "managed repeatability" in batch scoring and retraining scenarios, for example. For more information, see https://aka.ms/pl-first-pipeline.

Methods

disable

Set the published pipeline to 'Disabled' and unavailable to run.

enable

Set the published pipeline to 'Active' and available to run.

get

Get the published pipeline.

get_all

Get all published pipelines in the current workspace.

DEPRECATED: This method is being deprecated in favor of PublishedPipeline azureml.pipeline.core.graph.PublishedPipeline.list method.

get_graph

Get the graph of the PublishedPipeline.

get_step_names

Get the list of names of steps in the PublishedPipeline.

list

Get all published pipelines in the current workspace.

save

Save the Pipeline YAML to a file.

Currently, only pipelines consisting of ModuleSteps are supported for YAML exporting.

submit

Submit the published pipeline. This is equivalent to using azureml.core.Experiment.submit.

Returns the submitted PipelineRun. Use this object to monitor and view details of the run.

disable

Set the published pipeline to 'Disabled' and unavailable to run.

disable()

enable

Set the published pipeline to 'Active' and available to run.

enable()

get

Get the published pipeline.

get(workspace, id, _workflow_provider=None, _service_endpoint=None)

Parameters

workspace
Workspace

The workspace the published pipeline was created in.

id
str

The ID of the published pipeline.

_workflow_provider
<xref:_AevaWorkflowProvider object>
default value: None

The workflow provider.

_service_endpoint
str
default value: None

The service endpoint.

Returns

A PublishedPipeline object.

Return type

get_all

Get all published pipelines in the current workspace.

DEPRECATED: This method is being deprecated in favor of PublishedPipeline azureml.pipeline.core.graph.PublishedPipeline.list method.

get_all(workspace, active_only=True, _service_endpoint=None)

Parameters

workspace
Workspace

The workspace the published pipeline was created on.

active_only
bool
default value: True

Whether to return only published pipelines which are currently active.

_service_endpoint
str
default value: None

The service endpoint.

Returns

A list of PublishedPipeline objects.

Return type

get_graph

Get the graph of the PublishedPipeline.

get_graph(_workflow_provider=None)

Parameters

_workflow_provider
<xref:_AevaWorkflowProvider>
default value: None

The workflow provider.

Returns

The graph.

Return type

get_step_names

Get the list of names of steps in the PublishedPipeline.

get_step_names(_workflow_provider=None)

Parameters

_workflow_provider
<xref:_AevaWorkflowProvider>
default value: None

The workflow provider.

Returns

The list of the names of steps in the PublishedPipeline.

Return type

list

Get all published pipelines in the current workspace.

list(workspace, active_only=True, _service_endpoint=None)

Parameters

workspace
Workspace

The workspace the published pipeline was created in.

active_only
bool
default value: True

Whether to return only published pipelines which are currently active.

_service_endpoint
str
default value: None

The service endpoint.

Returns

A list of PublishedPipeline objects.

Return type

save

Save the Pipeline YAML to a file.

Currently, only pipelines consisting of ModuleSteps are supported for YAML exporting.

save(path=None, _workflow_provider=None)

Parameters

path
str
default value: None

The path to save the YAML to. If the path is a directory, the Pipeline YAML file is saved at path/pipeline_name.yml. If path is None, the current directory is used.

_workflow_provider
<xref:_AevaWorkflowProvider>
default value: None

The workflow provider.

Return type

submit

Submit the published pipeline. This is equivalent to using azureml.core.Experiment.submit.

Returns the submitted PipelineRun. Use this object to monitor and view details of the run.

submit(workspace, experiment_name, pipeline_parameters=None, _workflow_provider=None, _service_endpoint=None, parent_run_id=None, continue_on_step_failure=None)

Parameters

workspace
Workspace

The workspace to submit the published pipeline on.

experiment_name
str

The name of the experiment to submit to.

pipeline_parameters
dict
default value: None

A dictionary of parameters to assign new values {param name, param value}. See PipelineParameter for more details.

_workflow_provider
<xref:_AevaWorkflowProvider>
default value: None

The workflow provider.

_service_endpoint
str
default value: None

The service endpoint.

parent_run_id
str
default value: None

Optional run ID to set for the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to same experiment as this pipeline is being submitted to.

continue_on_step_failure
bool
default value: None

Whether to continue execution of other steps in the PipelineRun if a step fails, optional. If provided, will override the setting on the Pipeline.

Returns

The submitted pipeline run.

Return type

Attributes

continue_on_step_failure

Get the value of the continue_on_step_failure setting.

Returns

The value of the continue_on_step_failure setting.

Return type

description

Get the description of the published pipeline.

Returns

The description of the published pipeline.

Return type

str

endpoint

Get the REST endpoint URL for running a published pipeline.

Returns

The REST endpoint URL for running the published pipeline.

Return type

str

graph_id

Get the ID of the graph for this published pipeline.

Returns

The ID of the graph.

Return type

str

id

Get the published pipeline ID.

Returns

The ID of the published pipeline.

Return type

str

name

Get the name of the published pipeline.

Returns

The published pipeline name.

Return type

str

status

Get the status of the published pipeline.

Returns

The status of the published pipeline.

Return type

str

total_run_steps

Get the number of steps in the pipeline.

Returns

The number of steps in the pipeline.

Return type

int

version

Get the version of the published pipeline.

Returns

The version of the published pipeline.

Return type

str