PortDataReference class

Definition

Models data associated with an output of a completed StepRun.

A PortDataReference object can be used to download the output data which was produced by a StepRun. It can also be used as an step input in a future Pipeline.

PortDataReference(context, pipeline_run_id, data_reference, step_run=None)
Inheritance
builtins.object
PortDataReference

Parameters

context
_GraphContext

The graph context object.

pipeline_run_id
str

The id of the pipeline run which produced the output.

data_reference
DataReference

The data reference object.

step_run
azureml.pipeline.run.StepRun

The StepRun object which produced the data.

Remarks

A PortDataReference points to data which was produced by a completed StepRun.

Get a PortDataReference from a PipelineRun and download the output data as follows:


   from azureml.pipeline.core import PipelineRun, StepRun, PortDataReference

   pipeline_run = PipelineRun(experiment, "<pipeline_run_id>")
   step_run = pipeline_run.find_step_run("<node_name>")[0]
   port_data_reference = step_run.get_output_data("<output_name>")
   port_data_reference.download(local_path="path")

A PortDataReference can be used as an input to a step in a Pipeline, similar to using a DataReference.

An example to using a PortDataReference as a step input is as follows:


   from azureml.pipeline.core import PortDataReference, Pipeline, PipelineRun
   from azureml.pipeline.steps import PythonScriptStep

   pipeline_run = PipelineRun(experiment, "<pipeline_run_id>")
   step_run = pipeline_run.find_step_run("<node_name>")[0]
   port_data_reference = step_run.get_output_data("<output_name>")

   step_1 = PythonScriptStep(
       name='train',
       script_name="train.py",
       compute_target=compute,
       arguments=["--input", port_data_reference],
       inputs=[port_data_reference]
   )

   pipeline = Pipeline(workspace=workspace, steps=[step_1])

In this example, the newly constructed Pipeline will use the output which was created by the completed PipelineRun as an input to the train step.

Methods

as_download(input_name=None, path_on_compute=None, overwrite=None)

Consume the PortDataReference as a step input through the "download" mode.

as_input(input_name)

Create an InputPortBinding and specify an input name (but use default mode).

as_mount(input_name=None)

Consume the PortDataReference as a step input through the "mount" mode.

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Create input binding with this PortDataReference as the source.

download(local_path, overwrite=None, show_progress=None)

Download the data represented by the PortDataReference.

as_download(input_name=None, path_on_compute=None, overwrite=None)

Consume the PortDataReference as a step input through the "download" mode.

as_download(input_name=None, path_on_compute=None, overwrite=None)

Parameters

input_name
str

Specify a name for this input.

default value: None
path_on_compute
str

The path on the compute to download the data to.

default value: None
overwrite
bool

Use to indicate whether to overwrite existing data.

default value: None

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

as_input(input_name)

Create an InputPortBinding and specify an input name (but use default mode).

as_input(input_name)

Parameters

input_name
str

Use to specify a name for this input.

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

as_mount(input_name=None)

Consume the PortDataReference as a step input through the "mount" mode.

as_mount(input_name=None)

Parameters

input_name
str

Use to specify a name for this input.

default value: None

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Create input binding with this PortDataReference as the source.

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Parameters

input_name
str

The name of the input.

default value: None
mode
str

The mode to access the PortDataReference ("mount" or "download").

default value: None
path_on_compute
str

For "download" mode, the path on the compute the data will reside.

default value: None
overwrite
bool

For "download" mode, whether to overwrite existing data.

default value: None

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

download(local_path, overwrite=None, show_progress=None)

Download the data represented by the PortDataReference.

download(local_path, overwrite=None, show_progress=None)

Parameters

local_path
str

Local path to download to.

overwrite
bool, optional

If true, overwrite existing file, defaults to False.

default value: None
show_progress
bool, optional

Show the progress of download in the console, defaults to True.

default value: None

Returns

The number of files successfully downloaded.

Return type

int

Attributes

datastore

Get the Datastore associated with the PortDataReference.

Returns

The datastore object.

Return type

datastore_name

Get the name of the Datastore associated with the PortDataReference.

Returns

The datastore name.

Return type

str

name

Get the name of this data reference.

Returns

The data reference name.

Return type

str

path_on_datastore

Get the path on datastore for the PortDataReference.

Returns

The path on datastore.

Return type

str