PortDataReference class

Definition

Models data associated with an output of a completed StepRun.

A PortDataReference object can be used to download the output data which was produced by a StepRun. It can also be used as an step input in a future Pipeline.

PortDataReference(context, pipeline_run_id, data_reference, step_run=None)
Inheritance
builtins.object
PortDataReference

Parameters

context
_GraphContext

The graph context object.

pipeline_run_id
str

The id of the pipeline run which produced the output.

data_reference
DataReference

The data reference object.

step_run
azureml.pipeline.run.StepRun

The StepRun object which produced the data.

Remarks

A PortDataReference points to data which was produced by a completed StepRun.

Get a PortDataReference from a PipelineRun and download the output data as follows:


   from azureml.pipeline.core import PipelineRun, StepRun, PortDataReference

   pipeline_run = PipelineRun(experiment, "<pipeline_run_id>")
   step_run = pipeline_run.find_step_run("<node_name>")[0]
   port_data_reference = step_run.get_output_data("<output_name>")
   port_data_reference.download(local_path="path")

A PortDataReference can be used as an input to a step in a Pipeline, similar to using a DataReference.

An example to using a PortDataReference as a step input is as follows:


   from azureml.pipeline.core import PortDataReference, Pipeline, PipelineRun
   from azureml.pipeline.steps import PythonScriptStep

   pipeline_run = PipelineRun(experiment, "<pipeline_run_id>")
   step_run = pipeline_run.find_step_run("<node_name>")[0]
   port_data_reference = step_run.get_output_data("<output_name>")

   step_1 = PythonScriptStep(
       name='train',
       script_name="train.py",
       compute_target=compute,
       arguments=["--input", port_data_reference],
       inputs=[port_data_reference]
   )

   pipeline = Pipeline(workspace=workspace, steps=[step_1])

In this example, the newly constructed Pipeline will use the output which was created by the completed PipelineRun as an input to the train step.

Methods

as_download(input_name=None, path_on_compute=None, overwrite=None)

Consume the PortDataReference as a step input through the "download" mode.

as_input(input_name)

Create an InputPortBinding and specify an input name (but use default mode).

as_mount(input_name=None)

Consume the PortDataReference as a step input through the "mount" mode.

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Create input binding with this PortDataReference as the source.

download(local_path, overwrite=None, show_progress=None)

Download the data represented by the PortDataReference.

as_download(input_name=None, path_on_compute=None, overwrite=None)

Consume the PortDataReference as a step input through the "download" mode.

as_download(input_name=None, path_on_compute=None, overwrite=None)

Parameters

input_name
str

Specify a name for this input.

default value: None
path_on_compute
str

The path on the compute to download the data to.

default value: None
overwrite
bool

Use to indicate whether to overwrite existing data.

default value: None

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

as_input(input_name)

Create an InputPortBinding and specify an input name (but use default mode).

as_input(input_name)

Parameters

input_name
str

Use to specify a name for this input.

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

as_mount(input_name=None)

Consume the PortDataReference as a step input through the "mount" mode.

as_mount(input_name=None)

Parameters

input_name
str

Use to specify a name for this input.

default value: None

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Create input binding with this PortDataReference as the source.

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Parameters

input_name
str

The name of the input.

default value: None
mode
str

The mode to access the PortDataReference ("mount" or "download").

default value: None
path_on_compute
str

For "download" mode, the path on the compute the data will reside.

default value: None
overwrite
bool

For "download" mode, whether to overwrite existing data.

default value: None

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

download(local_path, overwrite=None, show_progress=None)

Download the data represented by the PortDataReference.

download(local_path, overwrite=None, show_progress=None)

Parameters

local_path
str

Local path to download to.

overwrite
bool or optional

If true, overwrite existing file, defaults to False.

default value: None
show_progress
bool or optional

Show the progress of download in the console, defaults to True.

default value: None

Returns

The number of files successfully downloaded.

Return type

int

Attributes

datastore

Get the Datastore associated with the PortDataReference.

Returns

The datastore object.

Return type

datastore_name

Get the name of the Datastore associated with the PortDataReference.

Returns

The datastore name.

Return type

str

path_on_datastore

Get the path on datastore for the PortDataReference.

Returns

The path on datastore.

Return type

str