PortDataReference Class

Models data associated with an output of a completed StepRun.

A PortDataReference object can be used to download the output data which was produced by a StepRun. It can also be used as an step input in a future Pipeline.

Inheritance
PortDataReference

Constructor

PortDataReference(context, pipeline_run_id, data_reference, step_run=None)

Parameters

context
<xref:_GraphContext>

The graph context object.

pipeline_run_id
str

The id of the pipeline run which produced the output.

data_reference
DataReference

The data reference object.

step_run
StepRun

The StepRun object which produced the data.

Remarks

A PortDataReference points to data which was produced by a completed StepRun.

Get a PortDataReference from a PipelineRun and download the output data as follows:


   from azureml.pipeline.core import PipelineRun, StepRun, PortDataReference

   pipeline_run = PipelineRun(experiment, "<pipeline_run_id>")
   step_run = pipeline_run.find_step_run("<node_name>")[0]
   port_data_reference = step_run.get_output_data("<output_name>")
   port_data_reference.download(local_path="path")

A PortDataReference can be used as an input to a step in a Pipeline, similar to using a DataReference.

An example to using a PortDataReference as a step input is as follows:


   from azureml.pipeline.core import PortDataReference, Pipeline, PipelineRun
   from azureml.pipeline.steps import PythonScriptStep

   pipeline_run = PipelineRun(experiment, "<pipeline_run_id>")
   step_run = pipeline_run.find_step_run("<node_name>")[0]
   port_data_reference = step_run.get_output_data("<output_name>")

   step_1 = PythonScriptStep(
       name='train',
       script_name="train.py",
       compute_target=compute,
       arguments=["--input", port_data_reference],
       inputs=[port_data_reference]
   )

   pipeline = Pipeline(workspace=workspace, steps=[step_1])

In this example, the newly constructed Pipeline will use the output which was created by the completed PipelineRun as an input to the train step.

Methods

as_download

Consume the PortDataReference as a step input through the "download" mode.

as_input

Create an InputPortBinding and specify an input name (but use default mode).

as_mount

Consume the PortDataReference as a step input through the "mount" mode.

create_input_binding

Create input binding with this PortDataReference as the source.

download

Download the data represented by the PortDataReference.

as_download

Consume the PortDataReference as a step input through the "download" mode.

as_download(input_name=None, path_on_compute=None, overwrite=None)

Parameters

input_name
str
default value: None

Specify a name for this input.

path_on_compute
str
default value: None

The path on the compute to download the data to.

overwrite
bool
default value: None

Use to indicate whether to overwrite existing data.

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

as_input

Create an InputPortBinding and specify an input name (but use default mode).

as_input(input_name)

Parameters

input_name
str

Use to specify a name for this input.

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

as_mount

Consume the PortDataReference as a step input through the "mount" mode.

as_mount(input_name=None)

Parameters

input_name
str
default value: None

Use to specify a name for this input.

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

create_input_binding

Create input binding with this PortDataReference as the source.

create_input_binding(input_name=None, mode=None, path_on_compute=None, overwrite=None)

Parameters

input_name
str
default value: None

The name of the input.

mode
str
default value: None

The mode to access the PortDataReference ("mount" or "download").

path_on_compute
str
default value: None

For "download" mode, the path on the compute the data will reside.

overwrite
bool
default value: None

For "download" mode, whether to overwrite existing data.

Returns

The InputPortBinding with this PortDataReference as the source.

Return type

download

Download the data represented by the PortDataReference.

download(local_path, overwrite=None, show_progress=None)

Parameters

local_path
str

Local path to download to.

overwrite
bool, <xref:optional>
default value: None

If true, overwrite existing file, defaults to False.

show_progress
bool, <xref:optional>
default value: None

Show the progress of download in the console, defaults to True.

Returns

The number of files successfully downloaded.

Return type

int

Attributes

datastore

Get the Datastore associated with the PortDataReference.

Returns

The datastore object.

Return type

datastore_name

Get the name of the Datastore associated with the PortDataReference.

Returns

The datastore name.

Return type

str

name

Get the name of this data reference.

Returns

The data reference name.

Return type

str

path_on_datastore

Get the path on datastore for the PortDataReference.

Returns

The path on datastore.

Return type

str