RunConfiguration class
Definition
Class for configuring an environment involving Experiment runs and Compute.
This class works with :submit(config, tags=None, **kwargs) to configure the execution environment for an experiment trial.
RunConfiguration(script=None, arguments=None, framework=None, communicator=None, conda_dependencies=None, _history_enabled=None, _path=None, _name=None)- Inheritance
-
builtins.objectazureml._base_sdk_common.abstract_run_config_element._AbstractRunConfigElementRunConfiguration
Parameters
- script
- str
The relative path to the python script file. The file path is relative to the source_directory passed to submit(config, tags=None, **kwargs).
- arguments
- <xref:azureml.core.runconfig.list[str]>
Command line arguments for the python script file.
- framework
- str
The supported frameworks are Python, PySpark, TensorFlow, and PyTorch.
- communicator
- str
The supported communicators are None, ParameterServer, OpenMpi, and IntelMpi Keep in mind that OpenMpi requires a custom image with OpenMpi installed. Use ParameterServer or OpenMpi for AmlCompute clusters. Use IntelMpi for distributed training jobs.
- environment
- Environment
The environment definition, This field configures the python environment. It can be configured to use an existing Python environment or configure to setup a temp environment for the experiment. The definition is also responsible for setting the required application dependencies.
- auto_prepare_environment
- bool
Deprecated setting that is no longer used.
- max_run_duration_seconds
- int
Maximum allowed time for the run. The system will attempt to automatically cancel the run, if it took longer than this value
- node_count
- int
Number of nodes to use for running job.
- history
- HistoryConfiguration
This section is used to disable and enable experiment history logging features.
- spark
- SparkConfiguration
When the platform is set to Pyspark, The spark configuration is used to set the default sparkconf for the submitted job.
- hdi
- HDIConfiguration
This attribute takes effect only when the target is set to an Azure HDI compute. The HDI Configuration is used to set the YARN deployment mode. It is defaulted to cluster mode.
- tensorflow
- TensorFlowConfiguration`
The attribute is used to configure the distributed tensorflow parameters. This parameter takes effect only when the framework is set to TensorFlow, and the communicator to ParameterServer. AmlCompute is the only supported compute for this configuration.
- mpi
- MPIConfiguration
The attribute is used to configure the distributed MPI job parameters. This parameter takes effect only when the framework is set to Python, and the communicator to OpenMpi or IntelMpi. AmlComppute is the only supported compute type for this configuration.
All the data sources are available to the run during execution based on each configuration. This parameter is a dict. For each item, the key is a name given to the data source. The value is a DataReferenceConfiguration.
- source_directory_data_store
- str
The attribute is used to configure the backing datastore for the project share.
- amlcompute
- AmlComputeConfiguration
The attribute is used to configure details of the compute target to be created during experiment. The configuration only takes effect when the target is set to "amlcompute"
- conda_dependencies
- CondaDependencies
When left at the default value of false, the system will create a python environment, which includes the packages specified in conda_dependencies. When set true, an existing python environment can be specified with the python_interpreter setting.
Remarks
The RunConfiguration encapsulates execution environment settings.
We build machine learning systems typically to solve a specific problem. For example, we might be interested in finding the best model that ranks web pages that might be served as search results corresponding to a query. Our search for the best machine learning model may have us try out different algorithms, or consider different parameter settings, etc.
In the Azure Machine Learning SDK, we use the concept of an experiment to capture the notion that different training runs are related by the problem that they're trying to solve. An Experiment then acts as a logical container for these training runs, making it easier to track progress across training runs, compare two training runs directly, etc.
The RunConfiguration object is used to codify the information necessary to submit a training run in an experiment. It captures both the shared structure of training runs that are designed to solve the same machine learning problem, as well as the differences in the configuration parameters (e.g. learning rate, loss function, etc) that distinguish distinct training runs from each other.
The configuration include:
Bundling the experiment source directory including the submitted script.
Setting the Command line arguments for the submitted script.
Configuring the path for the Python interpreter.
Obtain Conda configuration for to manage the application dependencies. The job submission process can use the configuration to
provision a temp Conda environment and launch the application within. The temp environments are cached and reused in subsequent runs.
Optional usage of Docker and custom base images.
Optional choice of submitting the experiment to multiple types of Azure compute.
Advanced runtime settings for common runtimes like spark and tensorflow.
For example users can submit a simple training script on the local machine using this code.
Methods
| delete(path, name) |
Delete a run configuration file. |
| load(path, name=None) |
Load a previously saved run configuration file from an on-disk file. If path points to a file then the RunConfiguration is loaded from that file. If path points to a directory, which should a project directory, then the RunConfiguration is loaded
from
|
| save(path=None, name=None, separate_environment_yaml=False) |
Save the RunConfiguration to a file on disk. If path is of <dir_path>/<file_name> format, where <dir_path> is a valid directory, then the RunConfiguration is saved at <dir_path>/<file_name> If path points to a directory, which should be a project directory, then the RunConfiguration is saved
at
This method is useful when editing the configuration manually or when sharing the configuration with the CLI. |
delete(path, name)
Delete a run configuration file.
delete(path, name)Parameters
- path
- str
A user selected root directory for run configurations. Typically this is the Git Repository or the python project root directory. The configuration is deleted from a sub directory named .azureml.
- name
- str
The configuration file name.
load(path, name=None)
Load a previously saved run configuration file from an on-disk file.
If path points to a file then the RunConfiguration is loaded from that file.
If path points to a directory, which should a project directory, then the RunConfiguration is loaded
from
load(path, name=None)Parameters
- path
- str
A user selected root directory for run configurations. Typically this is the Git Repository or the python project root directory. For backward compatibility, the configuration will also be loaded from .azureml or aml_config sub directory.
Returns
The run configuration object.
Return type
save(path=None, name=None, separate_environment_yaml=False)
Save the RunConfiguration to a file on disk.
If path is of <dir_path>/<file_name> format, where <dir_path> is a valid directory, then the RunConfiguration is saved at <dir_path>/<file_name>
If path points to a directory, which should be a project directory, then the RunConfiguration is saved
at
This method is useful when editing the configuration manually or when sharing the configuration with the CLI.
save(path=None, name=None, separate_environment_yaml=False)Parameters
- separate_environment_yaml
- bool
separate_environment_yaml=True saves the conda environment configuration file in a separate yaml file. The conda environment file name is named environment.yml
- path
- str
A user selected root directory for run configurations. Typically this is the Git Repository or the python project root directory. The configuration is saved to a sub directory named .azureml.
Return type
Attributes
auto_prepare_environment
Get auto_prepare_environment which is a deprecated and unused setting.
target
Get target.
Target refers to compute where the job is scheduled for execution. The default target is "local" refering to the local machine. Available cloud compute targets can be found using the function compute_targets
Returns
The target name
Return type
Feedback
Loading feedback...