InferenceConfig Class

Represents configuration settings for a custom environment used for deployment.

Inference configuration is an input parameter for Model deployment-related actions:

Inheritance
InferenceConfig

Constructor

InferenceConfig(entry_script, runtime=None, conda_file=None, extra_docker_file_steps=None, source_directory=None, enable_gpu=None, description=None, base_image=None, base_image_registry=None, cuda_version=None, environment=None)

Parameters

entry_script
str

The path to a local file that contains the code to run for the image.

runtime
str

The runtime to use for the image. Current supported runtimes are 'spark-py' and 'python'.

conda_file
str

The path to a local file containing a conda environment definition to use for the image.

extra_docker_file_steps
str

The path to a local file containing additional Docker steps to run when setting up image.

source_directory
str

The path to the folder that contains all files to create the image.

enable_gpu
bool

Indicates whether to enable GPU support in the image. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Defaults to False.

description
str

A description to give this image.

base_image
str

A custom image to be used as base image. If no base image is given then the base image will be used based off of given runtime parameter.

base_image_registry
ContainerRegistry

The image registry that contains the base image.

cuda_version
str

The Version of CUDA to install for images that need GPU support. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Supported versions are 9.0, 9.1, and 10.0. If enable_gpu is set, this defaults to '9.1'.

environment
Environment

An environment object to use for the deployment. The environment doesn't have to be registered.

Provide either this parameter, or the other parameters, but not both. The individual parameters will NOT serve as an override for the environment object. Exceptions include entry_script, source_directory, and description.

Remarks

The following sample shows how to create an InferenceConfig object and use it to deploy a model.


   from azureml.core.model import InferenceConfig
   from azureml.core.webservice import AciWebservice


   service_name = 'my-custom-env-service'

   inference_config = InferenceConfig(entry_script='score.py', environment=environment)
   aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)

   service = Model.deploy(workspace=ws,
                          name=service_name,
                          models=[model],
                          inference_config=inference_config,
                          deployment_config=aci_config,
                          overwrite=True)
   service.wait_for_deployment(show_output=True)

Full sample is available from https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/deploy-to-cloud/model-register-and-deploy.ipynb

Variables

entry_script
str

The path to a local file that contains the code to run for the image.

runtime
str

The runtime to use for the image. Current supported runtimes are 'spark-py' and 'python'.

conda_file
str

The path to a local file containing a conda environment definition to use for the image.

extra_docker_file_steps
str

The path to a local file containing additional Docker steps to run when setting up the image.

source_directory
str

The path to the folder that contains all files to create the image.

enable_gpu
bool

Indicates whether to enable GPU support in the image. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service.

azureml.core.model.InferenceConfig.description

A description to give this image.

base_image
str

A custom image to be used as base image. If no base image is given then the base image will be used based off of given runtime parameter.

base_image_registry
ContainerRegistry

The image registry that contains the base image.

cuda_version
str

The version of CUDA to install for images that need GPU support. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Supported versions are 9.0, 9.1, and 10.0. If enable_gpu is set, this defaults to '9.1'.

azureml.core.model.InferenceConfig.environment

An environment object to use for the deployment. The environment doesn't have to be registered.

Provide either this parameter, or the other parameters, but not both. The individual parameters will NOT serve as an override for the environment object. Exceptions include entry_script, source_directory, and description.

Methods

build_create_payload

Build the creation payload for the Container image.

build_profile_payload

Build the profiling payload for the Model package.

validate_configuration

Check that the specified configuration values are valid.

Raises a WebserviceException if validation fails.

validation_script_content

Check that the syntax of score script is valid with ast.parse.

Raises a UserErrorException if validation fails.

build_create_payload

Build the creation payload for the Container image.

build_create_payload(workspace, name, model_ids)

Parameters

workspace
Workspace

The workspace object to create the image in.

name
str

The name of the image.

model_ids
list[str]

A list of model IDs to package into the image.

Returns

The container image creation payload.

Return type

Exceptions

build_profile_payload

Build the profiling payload for the Model package.

build_profile_payload(profile_name, input_data=None, workspace=None, models=None, dataset_id=None, container_resource_requirements=None, description=None)

Parameters

profile_name
str

The name of the profiling run.

input_data
str
default value: None

The input data for profiling.

workspace
Workspace
default value: None

A Workspace object in which to profile the model.

models
list[Model]
default value: None

A list of model objects. Can be an empty list.

dataset_id
str
default value: None

Id associated with the dataset containing input data for the profiling run.

container_resource_requirements
ContainerResourceRequirements
default value: None

container resource requirements for the largest instance to which the model is to be deployed

description
str
default value: None

Description to be associated with the profiling run.

Returns

Model profile payload

Return type

Exceptions

validate_configuration

Check that the specified configuration values are valid.

Raises a WebserviceException if validation fails.

validate_configuration()

Exceptions

validation_script_content

Check that the syntax of score script is valid with ast.parse.

Raises a UserErrorException if validation fails.

validation_script_content()

Exceptions