Chainer Class

Represents an estimator for training in Chainer experiments.

DEPRECATED. Use the ScriptRunConfig object with your own defined environment or one of the Azure ML Chainer curated environments. For an introduction to configuring experiment runs with ScriptRunConfig, see Configure and submit training runs.

Supported versions: 5.1.0, 7.0.0

Initialize a Chainer estimator.

Inheritance
azureml.train.estimator._framework_base_estimator._FrameworkBaseEstimator
Chainer

Constructor

Chainer(source_directory, *, compute_target=None, vm_size=None, vm_priority=None, entry_script=None, script_params=None, node_count=1, process_count_per_node=1, distributed_backend=None, distributed_training=None, use_gpu=False, use_docker=True, custom_docker_base_image=None, custom_docker_image=None, image_registry_details=None, user_managed=False, conda_packages=None, pip_packages=None, conda_dependencies_file_path=None, pip_requirements_file_path=None, conda_dependencies_file=None, pip_requirements_file=None, environment_variables=None, environment_definition=None, inputs=None, source_directory_data_store=None, shm_size=None, resume_from=None, max_run_duration_seconds=None, framework_version=None, _enable_optimized_mode=False, _disable_validation=True, _show_lint_warnings=False, _show_package_warnings=False)

Parameters

Name Description
source_directory
Required
str

A local directory containing experiment configuration files.

compute_target
Required

The compute target where training will happen. This can either be an object or the string "local".

vm_size
Required
str

The VM size of the compute target that will be created for the training. Supported values: Any Azure VM size.

vm_priority
Required
str

The VM priority of the compute target that will be created for the training. If not specified, 'dedicated' is used.

Supported values: 'dedicated' and 'lowpriority'.

This takes effect only when the vm_size param is specified in the input.

entry_script
Required
str

The relative path to the file containing the training script.

script_params
Required

A dictionary of command-line arguments to pass to the training script specified in entry_script.

node_count
Required
int

The number of nodes in the compute target used for training. If greater than 1, an MPI distributed job will be run. Only the AmlCompute target is supported for distributed jobs.

process_count_per_node
Required
int

The number of processes per node. If greater than 1, an MPI distributed job will be run. Only the AmlCompute target is supported for distributed jobs.

distributed_backend
Required
str

The communication backend for distributed training.

DEPRECATED. Use the distributed_training parameter.

Supported values: 'mpi'.

'mpi': MPI/Horovod

This parameter is required when node_count or process_count_per_node > 1.

When node_count == 1 and process_count_per_node == 1, no backend will be used unless the backend is explicitly set. Only the AmlCompute target is supported for distributed training.

distributed_training
Required
Mpi

Parameters for running a distributed training job.

For running a distributed job with MPI backend, use the Mpi object to specify process_count_per_node.

use_gpu
Required

Specifies whether the environment to run the experiment should support GPUs. If true, GPU-based default Docker image will be used in the environment. If false, a CPU-based image will be used. Default docker images (CPU or GPU) will be used only if the custom_docker_image parameter is not set. This setting is used only in Docker-enabled compute targets.

use_docker
Required

Specifies whether the environment to run the experiment should be Docker-based.

custom_docker_base_image
Required
str

The name of the Docker image from which the image to use for training will be built.

DEPRECATED. Use the custom_docker_image parameter.

If not set, a default CPU-based image will be used as the base image.

custom_docker_image
Required
str

The name of the Docker image from which the image to use for training will be built. If not set, a default CPU-based image will be used as the base image.

image_registry_details
Required

The details of the Docker image registry.

user_managed
Required

Specifies whether Azure ML reuses an existing Python environment. If false, Azure ML will create a Python environment based on the conda dependencies specification.

conda_packages
Required

A list of strings representing conda packages to be added to the Python environment for the experiment.

pip_packages
Required

A list of strings representing pip packages to be added to the Python environment for the experiment.

conda_dependencies_file_path
Required
str

The relative path to the conda dependencies yaml file. If specified, Azure ML will not install any framework related packages. DEPRECATED. Use the conda_dependencies_file parameter.

pip_requirements_file_path
Required
str

The relative path to the pip requirements text file. This can be provided in combination with the pip_packages parameter. DEPRECATED. Use the pip_requirements_file parameter.

conda_dependencies_file
Required
str

The relative path to the conda dependencies yaml file. If specified, Azure ML will not install any framework related packages.

pip_requirements_file
Required
str

The relative path to the pip requirements text file. This can be provided in combination with the pip_packages parameter.

environment_variables
Required

A dictionary of environment variables names and values. These environment variables are set on the process where user script is being executed.

environment_definition
Required

The environment definition for the experiment. It includes PythonSection, DockerSection, and environment variables. Any environment option not directly exposed through other parameters to the Estimator construction can be set using this parameter. If this parameter is specified, it will take precedence over other environment-related parameters like use_gpu, custom_docker_image, conda_packages, or pip_packages. Errors will be reported on invalid combinations.

inputs
Required

A list of DataReference or DatasetConsumptionConfig objects to use as input.

source_directory_data_store
Required

The backing datastore for project share.

shm_size
Required
str

The size of the Docker container's shared memory block. If not set, the default azureml.core.environment._DEFAULT_SHM_SIZE is used. For more information, see Docker run reference.

resume_from
Required

The data path containing the checkpoint or model files from which to resume the experiment.

max_run_duration_seconds
Required
int

The maximum allowed time for the run. Azure ML will attempt to automatically cancel the run if it take longer than this value.

framework_version
Required
str

The Chainer version to be used for executing training code. Chainer.get_supported_versions() returns a list of the versions supported by the current SDK.

source_directory
Required
str

A local directory containing experiment configuration files.

compute_target
Required

The compute target where training will happen. This can either be an object or the string "local".

vm_size
Required
str

The VM size of the compute target that will be created for the training. Supported values: Any Azure VM size.

vm_priority
Required
str

The VM priority of the compute target that will be created for the training. If not specified, it will be defaulted to 'dedicated'.

Supported values: 'dedicated' and 'lowpriority'.

This takes effect only when the vm_size param is specified in the input.

entry_script
Required
str

The relative path to the file containing the training script.

script_params
Required

A dictionary of command-line arguments to pass to the training script specified in entry_script.

node_count
Required
int

The number of nodes in the compute target used for training. If greater than 1, an MPI distributed job will be run. Only the AmlCompute target is supported for distributed jobs.

process_count_per_node
Required
int

The number of processes per node. If greater than 1, an MPI distributed job will be run. Only AmlCompute target compute target is supported for distributed jobs.

distributed_backend
Required
str

The communication backend for distributed training.

DEPRECATED. Use the distributed_training parameter.

Supported values: 'mpi'.

'mpi': MPI/Horovod

This parameter is required when node_count or process_count_per_node > 1.

When node_count == 1 and process_count_per_node == 1, no backend will be used unless the backend is explicitly set. Only the AmlCompute target is supported for distributed training.

distributed_training
Required
Mpi

Parameters for running a distributed training job.

For running a distributed job with MPI backend, use the Mpi object to specify process_count_per_node.

use_gpu
Required

Specifies whether the environment to run the experiment should support GPUs. If true, a GPU-based default Docker image will be used in the environment. If false, a CPU-based image will be used. Default Docker images (CPU or GPU) will be used only if the custom_docker_image parameter is not set. This setting is used only in Docker-enabled compute targets.

use_docker
Required

Specifies whether the environment to run the experiment should be Docker-based.

custom_docker_base_image
Required
str

The name of the Docker image from which the image to use for training will be built.

DEPRECATED. Use the custom_docker_image parameter.

If not set, a default CPU-based image will be used as the base image.

custom_docker_image
Required
str

The name of the Docker image from which the image to use for training will be built. If not set, a default CPU-based image will be used as the base image.

image_registry_details
Required

The details of the Docker image registry.

user_managed
Required

Specifies whether Azure ML reuses an existing Python environment. If false, Azure ML will create a Python environment based on the conda dependencies specification.

conda_packages
Required

A list of strings representing conda packages to be added to the Python environment for the experiment.

pip_packages
Required

A list of strings representing pip packages to be added to the Python environment for the experiment.

conda_dependencies_file_path
Required
str

The relative path to the conda dependencies yaml file. If specified, Azure ML will not install any framework related packages. DEPRECATED. Use the conda_dependencies_file parameter.

pip_requirements_file_path
Required
str

The relative path to the pip requirements text file. This can be provided in combination with the pip_packages parameter. DEPRECATED. Use the pip_requirements_file parameter.

conda_dependencies_file
Required
str

The relative path to the conda dependencies yaml file. If specified, Azure ML will not install any framework related packages.

pip_requirements_file
Required
str

The relative path to the pip requirements text file. This can be provided in combination with the pip_packages parameter.

environment_variables
Required

A dictionary of environment variables names and values. These environment variables are set on the process where user script is being executed.

environment_definition
Required

The environment definition for the experiment. It includes PythonSection, DockerSection, and environment variables. Any environment option not directly exposed through other parameters to the Estimator construction can be set using this parameter. If this parameter is specified, it will take precedence over other environment-related parameters like use_gpu, custom_docker_image, conda_packages, or pip_packages. Errors will be reported on invalid combinations.

inputs
Required

A list of azureml.data.data_reference.DataReference objects to use as input.

source_directory_data_store
Required

The backing datastore for project share.

shm_size
Required
str

The size of the Docker container's shared memory block. If not set, the default azureml.core.environment._DEFAULT_SHM_SIZE is used. For more information, see Docker run reference.

resume_from
Required

The data path containing the checkpoint or model files from which to resume the experiment.

max_run_duration_seconds
Required
int

The maximum allowed time for the run. Azure ML will attempt to automatically cancel the run if it take longer than this value.

framework_version
Required
str

The Chainer version to be used for executing training code. Chainer.get_supported_versions() returns a list of the versions supported by the current SDK.

_enable_optimized_mode
Required

Enable incremental environment build with pre-built framework images for faster environment preparation. A pre-built framework image is built on top of Azure ML default CPU/GPU base images with framework dependencies pre-installed.

_disable_validation
Required

Disable script validation before run submission. The default is True.

_show_lint_warnings
Required

Show script linting warnings. The default is False.

_show_package_warnings
Required

Show package validation warnings. The default is False.

Remarks

When submitting a training job, Azure ML runs your script in a conda environment within a Docker container. Chainer containers have the following dependencies installed.

Dependencies | Chainer 5.1.0 | Chainer 7.0.0 | —————————- | —————– | ————— | Python | 3.6.2 | 3.6.2 | CUDA (GPU image only) | 9.0 | 9.0 | cuDNN (GPU image only) | 7.6.3 | 7.6.3 | NCCL (GPU image only) | 2.4.8 | 2.4.8 | azureml-defaults | Latest | Latest | IntelMpi | 2018.3.222 | 2018.3.222 | horovod | 0.15.2 | 0.15.2 | miniconda | 4.5.11 | 4.5.11 | chainer | 5.1.0 | 7.0.0 | cupy-cuda90 (GPU image only) | 5.2.0 | 7.0.0 | git | 2.7.4 | 2.7.4 |

The Docker images extend Ubuntu 16.04.

To install additional dependencies, you can either use the pip_packages or conda_packages parameter. Or, you can specify the pip_requirements_file or conda_dependencies_file parameter. Alternatively, you can build your own image, and pass the custom_docker_image parameter to the estimator constructor.

For more information about Docker containers used in Chainer training, see https://github.com/Azure/AzureML-Containers.

Attributes

DEFAULT_VERSION

DEFAULT_VERSION = '5.1.0'

FRAMEWORK_NAME

FRAMEWORK_NAME = 'Chainer'