az ml endpoint realtime

Manage operationalized realtime endpoints.

Commands

az ml endpoint realtime create-version

Create a version for realtime endpoint in the workspace.

az ml endpoint realtime delete

Delete a realtime endpoint and its version from the workspace.

az ml endpoint realtime delete-version

Delete a version for realtime endpoint in the workspace.

az ml endpoint realtime get-access-token

Get a token to issue requests a realtime endpoint.

az ml endpoint realtime get-keys

Get keys to issue requests against a realtime endpoint.

az ml endpoint realtime get-logs

Get logs for a realtime endpoint.

az ml endpoint realtime list

List realtime endpoints in the workspace.

az ml endpoint realtime regen-key

Regenerate keys for a realtime endpoint.

az ml endpoint realtime run

Run a realtime endpoint in the workspace.

az ml endpoint realtime show

Show details for a realtime endpoint in the workspace.

az ml endpoint realtime update

Update a realtime endpoint in the workspace.

az ml endpoint realtime update-version

Update a version for realtime endpoint in the workspace.

az ml endpoint realtime create-version

Create a version for realtime endpoint in the workspace.

az ml endpoint realtime create-version --name
                                       --version-name
                                       [--add-property]
                                       [--add-tag]
                                       [--ar]
                                       [--as]
                                       [--at]
                                       [--autoscale-max-replicas]
                                       [--autoscale-min-replicas]
                                       [--cc]
                                       [--ccl]
                                       [--cf]
                                       [--collect-model-data]
                                       [--cvt]
                                       [--dc]
                                       [--description]
                                       [--ed]
                                       [--entry-script]
                                       [--environment-name]
                                       [--environment-version]
                                       [--failure-threshold]
                                       [--gb]
                                       [--gbl]
                                       [--gc]
                                       [--ic]
                                       [--id]
                                       [--is-default]
                                       [--max-request-wait-time]
                                       [--model]
                                       [--model-metadata-file]
                                       [--no-wait]
                                       [--nr]
                                       [--path]
                                       [--period-seconds]
                                       [--replica-max-concurrent-requests]
                                       [--resource-group]
                                       [--scoring-timeout-ms]
                                       [--sd]
                                       [--st]
                                       [--subscription-id]
                                       [--timeout-seconds]
                                       [--tp]
                                       [--workspace-name]
                                       [-v]

Required Parameters

--name -n

The endpoint name.

--version-name --vn

The version name to create in an Endpoint.

Optional Parameters

--add-property

Key/value property to add (e.g. key=value ). Multiple properties can be specified with multiple --add-property options.

--add-tag

Key/value tag to add (e.g. key=value ). Multiple tags can be specified with multiple --add-tag options.

--ar --autoscale-refresh-seconds

How often the autoscaler should attempt to scale this Webservice. Defaults to 1.

--as --autoscale-enabled

Whether or not to enable autoscaling for this Webservice. Defaults to True if num_replicas is None.

--at --autoscale-target-utilization

The target utilization (in percent out of 100) the autoscaler should attempt to maintain for this Webservice. Defaults to 70.

--autoscale-max-replicas --ma

The maximum number of containers to use when autoscaling this Webservice. Defaults to 10.

--autoscale-min-replicas --mi

The minimum number of containers to use when autoscaling this Webservice. Defaults to 1.

--cc --cpu-cores

The number of cpu cores to allocate for this Webservice. Can be a decimal. Defaults to 0.1.

--ccl --cpu-cores-limit

The max number of CPU cores this Webservice is allowed to use. Can be a decimal.

--cf --conda-file

Path to local file containing a conda environment definition to use for the image.

--collect-model-data --md

Whether or not to enable model data collection for this Webservice. Defaults to False.

--cvt --is-control-version-type

Whether or not this is the control version in an Endpoint. Defaults to False.

--dc --deploy-config-file

Path to a JSON file containing deployment metadata.

--description -d

Description of the service.

--ed --environment-directory

Directory for Azure Machine Learning Environment for deployment. It is the same directory path as provided in 'az ml environment scaffold' command.

--entry-script --es

Path to local file that contains the code to run for service (relative path from source_directory if one is provided).

--environment-name -e

Name of Azure Machine Learning Environment for deployment.

--environment-version --ev

Version of an existing Azure Machine Learning Environment for deployment.

--failure-threshold --ft

When a Pod starts and the liveness probe fails, Kubernetes will try --failure-threshold times before giving up. Defaults to 3. Minimum value is 1.

--gb --memory-gb

The amount of memory (in GB) to allocate for this Webservice. Can be a decimal.

--gbl --memory-gb-limit

The max amount of memory (in GB) this Webservice is allowed to use. Can be a decimal.

--gc --gpu-cores

The number of gpu cores to allocate for this Webservice. Default is 1.

--ic --inference-config-file

Path to a JSON or YAML file containing inference configuration.

--id --initial-delay-seconds

Number of seconds after the container has started before liveness probes are initiated. Defaults to 310.

--is-default

Whether or not this is the default version in an Endpoint. Defaults to False.

--max-request-wait-time --mr

The maximum amount of time a request will stay in the queue (in milliseconds) before returning a 503 error. Defaults to 500.

--model -m

The ID of the model to be deployed. Multiple models can be specified with additional -m arguments. Models need to be registered first.

--model-metadata-file -f

Path to a JSON file containing model registration metadata. Multiple models can be provided using multiple -f parameters.

--no-wait

Flag to not wait for asynchronous calls.

--nr --num-replicas

The number of containers to allocate for this Webservice. No default, if this parameter is not set then the autoscaler is enabled by default.

--path

Path to a project folder. Default: current directory.

--period-seconds --ps

How often (in seconds) to perform the liveness probe. Default to 10 seconds. Minimum value is 1.

--replica-max-concurrent-requests --rm

The number of maximum concurrent requests per node to allow for this Webservice. Defaults to 1.

--resource-group -g

Resource group corresponding to the provided workspace.

--scoring-timeout-ms --tm

A timeout to enforce for scoring calls to this Webservice. Defaults to 60000.

--sd --source-directory

Path to folders that contain all files to create the image.

--st --success-threshold

Minimum consecutive successes for the liveness probe to be considered successful after having failed. Defaults to 1. Minimum value is 1.

--subscription-id

Specifies the subscription Id.

--timeout-seconds --ts

Number of seconds after which the liveness probe times out. Defaults to 2 second. Minimum value is 1.

--tp --traffic-percentile

The traffic percentile this version takes in Endpoint.

--workspace-name -w

Name of the workspace containing the service to update.

-v

Verbosity flag.

az ml endpoint realtime delete

Delete a realtime endpoint and its version from the workspace.

az ml endpoint realtime delete --name
                               [--path]
                               [--resource-group]
                               [--subscription-id]
                               [--workspace-name]
                               [-v]

Required Parameters

--name -n

The endpoint name to delete.

Optional Parameters

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to delete.

-v

Verbosity flag.

az ml endpoint realtime delete-version

Delete a version for realtime endpoint in the workspace.

az ml endpoint realtime delete-version --name
                                       --version-name
                                       [--no-wait]
                                       [--path]
                                       [--resource-group]
                                       [--subscription-id]
                                       [--workspace-name]
                                       [-v]

Required Parameters

--name -n

The endpoint name.

--version-name --vn

The version name to delete.

Optional Parameters

--no-wait

Flag to not wait for asynchronous calls.

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to delete.

-v

Verbosity flag.

az ml endpoint realtime get-access-token

Get a token to issue requests a realtime endpoint.

az ml endpoint realtime get-access-token --name
                                         [--path]
                                         [--resource-group]
                                         [--subscription-id]
                                         [--workspace-name]
                                         [-v]

Required Parameters

--name -n

Endpoint Name.

Optional Parameters

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to show.

-v

Verbosity flag.

az ml endpoint realtime get-keys

Get keys to issue requests against a realtime endpoint.

az ml endpoint realtime get-keys --name
                                 [--path]
                                 [--resource-group]
                                 [--subscription-id]
                                 [--workspace-name]
                                 [-v]

Required Parameters

--name -n

Endpoint Name.

Optional Parameters

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to show.

-v

Verbosity flag.

az ml endpoint realtime get-logs

Get logs for a realtime endpoint.

az ml endpoint realtime get-logs --name
                                 [--init]
                                 [--num_lines]
                                 [--path]
                                 [--resource-group]
                                 [--subscription-id]
                                 [--workspace-name]
                                 [-v]

Required Parameters

--name -n

Endpoint Name.

Optional Parameters

--init -i

Get logs of init container instead of the scoring container.

--num_lines -l

Number of log lines to return from tail (default is 5000).

default value: 5000
--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to show.

-v

Verbosity flag.

az ml endpoint realtime list

List realtime endpoints in the workspace.

az ml endpoint realtime list [--compute-type]
                             [--model-id]
                             [--model-name]
                             [--path]
                             [--property]
                             [--resource-group]
                             [--subscription-id]
                             [--tag]
                             [--workspace-name]
                             [-v]

Optional Parameters

--compute-type -c

If provided, will only show services that have the specified compute type. (Options are 'ACI', 'AKS', 'AKSENDPOINT').

--model-id

If provided, will only show services that have the specified model id.

--model-name

If provided, will only show services that have the specified model name.

--path

Path to a project folder. Default: current directory.

--property

If provided, will filter based on the provided key/value (e.g. key, or key=value). Multiple properties can be specified with multiple --property options.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--tag

If provided, will filter based on the provided key/value (e.g. key, or key=value). Multiple tags can be specified with multiple --tag options.

--workspace-name -w

Name of the workspace containing the endpoints to list.

-v

Verbosity flag.

az ml endpoint realtime regen-key

Regenerate keys for a realtime endpoint.

az ml endpoint realtime regen-key --key
                                  --name
                                  [--path]
                                  [--resource-group]
                                  [--set-key]
                                  [--subscription-id]
                                  [--workspace-name]
                                  [-v]

Required Parameters

--key -k

Which key to regenerate, if regen is specified. Options: Primary, Secondary.

--name -n

Endpoint Name.

Optional Parameters

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--set-key -s

Provide auth value for the specified key.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to show.

-v

Verbosity flag.

az ml endpoint realtime run

Run a realtime endpoint in the workspace.

az ml endpoint realtime run --name
                            [--input-data]
                            [--path]
                            [--resource-group]
                            [--subscription-id]
                            [--workspace-name]
                            [-v]

Required Parameters

--name -n

The endpoint name to score against.

Optional Parameters

--input-data -d

The data to use for calling the endpoint.

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to run.

-v

Verbosity flag.

az ml endpoint realtime show

Show details for a realtime endpoint in the workspace.

az ml endpoint realtime show --name
                             [--path]
                             [--resource-group]
                             [--subscription-id]
                             [--workspace-name]
                             [-v]

Required Parameters

--name -n

Name of the endpoint to show.

Optional Parameters

--path

Path to a project folder. Default: current directory.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Name of the workspace containing the endpoint to show.

-v

Verbosity flag.

az ml endpoint realtime update

Update a realtime endpoint in the workspace.

az ml endpoint realtime update --name
                               [--add-property]
                               [--add-tag]
                               [--ae]
                               [--ai]
                               [--description]
                               [--no-wait]
                               [--path]
                               [--remove-tag]
                               [--resource-group]
                               [--subscription-id]
                               [--token-auth-enabled]
                               [--workspace-name]
                               [-v]

Required Parameters

--name -n

The endpoint name to update.

Optional Parameters

--add-property

Key/value property to add (e.g. key=value ). Multiple properties can be specified with multiple --add-property options.

--add-tag

Key/value tag to add (e.g. key=value ). Multiple tags can be specified with multiple --add-tag options.

--ae --auth-enabled

Whether or not to enable key auth for this Endpoint. Defaults to False.

--ai --enable-app-insights

Whether or not to enable AppInsights for this Endpoint. Defaults to False.

--description -d

Description of the endpoint.

--no-wait

Flag to not wait for asynchronous calls.

--path

Path to a project folder. Default: current directory.

--remove-tag

Key of tag to remove. Multiple tags can be specified with multiple --remove-tag options.

--resource-group -g

Resource group corresponding to the provided workspace.

--subscription-id

Specifies the subscription Id.

--token-auth-enabled

Whether or not to enable token auth for this Endpoint. Defaults to False.

--workspace-name -w

Name of the workspace containing the Endpoint to update.

-v

Verbosity flag.

az ml endpoint realtime update-version

Update a version for realtime endpoint in the workspace.

az ml endpoint realtime update-version --name
                                       --version-name
                                       [--add-property]
                                       [--add-tag]
                                       [--ar]
                                       [--as]
                                       [--at]
                                       [--autoscale-max-replicas]
                                       [--autoscale-min-replicas]
                                       [--cc]
                                       [--ccl]
                                       [--cf]
                                       [--collect-model-data]
                                       [--cvt]
                                       [--dc]
                                       [--description]
                                       [--ed]
                                       [--entry-script]
                                       [--environment-name]
                                       [--environment-version]
                                       [--failure-threshold]
                                       [--gb]
                                       [--gbl]
                                       [--gc]
                                       [--ic]
                                       [--id]
                                       [--is-default]
                                       [--max-request-wait-time]
                                       [--model]
                                       [--model-metadata-file]
                                       [--no-wait]
                                       [--nr]
                                       [--path]
                                       [--period-seconds]
                                       [--replica-max-concurrent-requests]
                                       [--resource-group]
                                       [--scoring-timeout-ms]
                                       [--sd]
                                       [--st]
                                       [--subscription-id]
                                       [--timeout-seconds]
                                       [--tp]
                                       [--workspace-name]
                                       [-v]

Required Parameters

--name -n

The endpoint name.

--version-name --vn

The version name to create in an Endpoint.

Optional Parameters

--add-property

Key/value property to add (e.g. key=value ). Multiple properties can be specified with multiple --add-property options.

--add-tag

Key/value tag to add (e.g. key=value ). Multiple tags can be specified with multiple --add-tag options.

--ar --autoscale-refresh-seconds

How often the autoscaler should attempt to scale this Webservice. Defaults to 1.

--as --autoscale-enabled

Whether or not to enable autoscaling for this Webservice. Defaults to True if num_replicas is None.

--at --autoscale-target-utilization

The target utilization (in percent out of 100) the autoscaler should attempt to maintain for this Webservice. Defaults to 70.

--autoscale-max-replicas --ma

The maximum number of containers to use when autoscaling this Webservice. Defaults to 10.

--autoscale-min-replicas --mi

The minimum number of containers to use when autoscaling this Webservice. Defaults to 1.

--cc --cpu-cores

The number of cpu cores to allocate for this Webservice. Can be a decimal. Defaults to 0.1.

--ccl --cpu-cores-limit

The max number of CPU cores this Webservice is allowed to use. Can be a decimal.

--cf --conda-file

Path to local file containing a conda environment definition to use for the image.

--collect-model-data --md

Whether or not to enable model data collection for this Webservice. Defaults to False.

--cvt --is-control-version-type

Whether or not this is the control version in an Endpoint. Defaults to False.

--dc --deploy-config-file

Path to a JSON file containing deployment metadata.

--description -d

Description of the service.

--ed --environment-directory

Directory for Azure Machine Learning Environment for deployment. It is the same directory path as provided in 'az ml environment scaffold' command.

--entry-script --es

Path to local file that contains the code to run for service (relative path from source_directory if one is provided).

--environment-name -e

Name of Azure Machine Learning Environment for deployment.

--environment-version --ev

Version of an existing Azure Machine Learning Environment for deployment.

--failure-threshold --ft

When a Pod starts and the liveness probe fails, Kubernetes will try --failure-threshold times before giving up. Defaults to 3. Minimum value is 1.

--gb --memory-gb

The amount of memory (in GB) to allocate for this Webservice. Can be a decimal.

--gbl --memory-gb-limit

The max amount of memory (in GB) this Webservice is allowed to use. Can be a decimal.

--gc --gpu-cores

The number of gpu cores to allocate for this Webservice. Default is 1.

--ic --inference-config-file

Path to a JSON or YAML file containing inference configuration.

--id --initial-delay-seconds

Number of seconds after the container has started before liveness probes are initiated. Defaults to 310.

--is-default

Whether or not this is the default version in an Endpoint. Defaults to False.

--max-request-wait-time --mr

The maximum amount of time a request will stay in the queue (in milliseconds) before returning a 503 error. Defaults to 500.

--model -m

The ID of the model to be deployed. Multiple models can be specified with additional -m arguments. Models need to be registered first.

--model-metadata-file -f

Path to a JSON file containing model registration metadata. Multiple models can be provided using multiple -f parameters.

--no-wait

Flag to not wait for asynchronous calls.

--nr --num-replicas

The number of containers to allocate for this Webservice. No default, if this parameter is not set then the autoscaler is enabled by default.

--path

Path to a project folder. Default: current directory.

--period-seconds --ps

How often (in seconds) to perform the liveness probe. Default to 10 seconds. Minimum value is 1.

--replica-max-concurrent-requests --rm

The number of maximum concurrent requests per node to allow for this Webservice. Defaults to 1.

--resource-group -g

Resource group corresponding to the provided workspace.

--scoring-timeout-ms --tm

A timeout to enforce for scoring calls to this Webservice. Defaults to 60000.

--sd --source-directory

Path to folders that contain all files to create the image.

--st --success-threshold

Minimum consecutive successes for the liveness probe to be considered successful after having failed. Defaults to 1. Minimum value is 1.

--subscription-id

Specifies the subscription Id.

--timeout-seconds --ts

Number of seconds after which the liveness probe times out. Defaults to 2 second. Minimum value is 1.

--tp --traffic-percentile

The traffic percentile this version takes in Endpoint.

--workspace-name -w

Name of the workspace containing the service to update.

-v

Verbosity flag.