OutputDatasetConfig Class

Represent how to copy the output of a job to a remote storage location and be promoted to a Dataset.

This is the base class used to represent how to copy the output of a job to a remote storage location, whether to register it as a named and versioned Dataset, and whether to apply any additional transformations to the Dataset that was created.

You should not be creating instances of this class directly but instead should use the appropriate subclass.

Initialize a OutputDatasetConfig.

Inheritance

builtins.object

OutputDatasetConfig

Constructor

OutputDatasetConfig(mode, name=None, **kwargs)

Parameters

mode

str

Required

The mode in which to copy the output to the remote storage.

name

str

Required

The name of the output specific to the run it will be produced in.

Methods

as_input	Specify how to consume the output as an input in subsequent pipeline steps.
register_on_complete	Register the output as a new version of a named Dataset after the run has ran. If there are no datasets registered under the specified name, a new Dataset with the specified name will be registered. If there is a dataset registered under the specified name, then a new version will be added to this dataset.

as_input

Specify how to consume the output as an input in subsequent pipeline steps.

as_input(name=None)

Parameters

name

str

default value: None

The name of the input specific to the run.

Returns

A DatasetConsumptionConfig instance describing how to deliver the input data.

Return type

DatasetConsumptionConfig

register_on_complete

Register the output as a new version of a named Dataset after the run has ran.

If there are no datasets registered under the specified name, a new Dataset with the specified name will be registered. If there is a dataset registered under the specified name, then a new version will be added to this dataset.

register_on_complete(name, description=None, tags=None)

Parameters

name

str

Required

The Dataset name to register the output under.

description

str

Required

The description for the Dataset.

tags

dict[str, str]

Required

A list of tags to be assigned to the Dataset.

Returns

A new OutputDatasetConfig instance with the registration information.

Return type

OutputDatasetConfig

Attributes

name

Name of the output.

Returns

Name of the output.