PythonScriptStep class

Definition

Creates an Azure ML Pipeline step that runs Python script.

For an example of using PythonScriptStep, see the notebook https://aka.ms/pl-get-started.

Supported values: 'NodeCount', 'MpiProcessCountPerNode', 'TensorflowWorkerCount', 'TensorflowParameterServerCount'

PythonScriptStep(script_name, name=None, arguments=None, compute_target=None, runconfig=None, runconfig_pipeline_params=None, inputs=None, outputs=None, params=None, source_directory=None, allow_reuse=True, version=None, hash_paths=None)
Inheritance
azureml.pipeline.core._python_script_step_base._PythonScriptStepBase
PythonScriptStep

Parameters

inputs
list[InputPortBinding or DataReference or PortDataReference or PipelineData or azureml.pipeline.core.pipeline_output_dataset.PipelineOutputDataset or Dataset or DatasetDefinition or DatasetConsumptionConfig or PipelineDataset]

A list of input port bindings.

outputs
list[PipelineData or azureml.pipeline.core.pipeline_output_dataset.PipelineOutputDataset or OutputPortBinding]

A list of output port bindings.

params
dict

A dictionary of name-value pairs registered as environment variables with "AML_PARAMETER_".

source_directory
str

A folder that contains Python script, conda env, and other resources used in the step.

allow_reuse
bool

Indicates whether the step should reuse previous results when re-run with the same settings. Reuse is enabled by default. If the step contents (scripts/dependencies) as well as inputs and parameters remain unchanged, the output from the previous run of this step is reused. When reusing the step, instead of submitting the job to compute, the results from the previous run are immediately made available to any subsequent steps. If you use Azure Machine Learning datasets as inputs, reuse is determined by whether the dataset's definition has changed, not by whether the underlying data has changed.

version
str

An optional version tag to denote a change in functionality for the step.

hash_paths
list

DEPRECATED: no longer needed.

A list of paths to hash when checking for changes to the step contents. If there are no changes detected, the pipeline will reuse the step contents from a previous run. By default, the contents of source_directory is hashed except for files listed in .amlignore or .gitignore.

Remarks

A PythonScriptStep is a basic, built-in step to run a Python Script on a compute target. It takes a script name and other optional parameters like arguments for the script, compute target, inputs and outputs. If no compute target is specified, the default compute target for the workspace is used. You can also use a RunConfiguration to specify requirements for the PythonScriptStep, such as conda dependencies and docker image.

The best practice for working with PythonScriptStep is to use a separate folder for scripts and any dependent files associated with the step, and specify that folder with the source_directory parameter. Folowing this best practice has two benefits. First, it helps reduce the size of the snapshot created for the step because only what is needed for the step is snapshotted. Second, the step's output from a previous run can be reused if there are no changes to the source_directory that would trigger a re-upload of the snaphot.

The following code example shows using a PythonScriptStep in a machine learning training scenario. For more details on this example, see https://aka.ms/pl-first-pipeline.


   from azureml.pipeline.steps import PythonScriptStep

   trainStep = PythonScriptStep(
       script_name="train.py",
       arguments=["--input", blob_input_data, "--output", output_data1],
       inputs=[blob_input_data],
       outputs=[output_data1],
       compute_target=compute_target,
       source_directory=project_folder
   )

Methods

create_node(graph, default_datastore, context)

Create a node for PythonScriptStep and add it to the specified graph.

This method is not intended to be used directly. When a pipeline is instantiated with this step, Azure ML automatically passes the parameters required through this method so that step can be added to a pipeline graph that represents the workflow.

create_node(graph, default_datastore, context)

Create a node for PythonScriptStep and add it to the specified graph.

This method is not intended to be used directly. When a pipeline is instantiated with this step, Azure ML automatically passes the parameters required through this method so that step can be added to a pipeline graph that represents the workflow.

create_node(graph, default_datastore, context)

Parameters

graph
Graph

The graph object to add the node to.

default_datastore
AbstractAzureStorageDatastore or AzureDataLakeDatastore

The default datastore.

context
_GraphContext

The graph context.

Returns

The created node.

Return type