Create and use managed online endpoints (preview) in the studio
Learn how to use the studio to create and manage your managed online endpoints (preview) in Azure Machine Learning. Use managed online endpoints to streamline production-scale deployments. For more information on managed online endpoints, see What are endpoints.
In this article, you learn how to:
- Create a managed online endpoint
- View managed online endpoints
- Add a deployment to a managed online endpoint
- Update managed online endpoints
- Delete managed online endpoints and deployments
Important
This feature is currently in public preview. This preview version is provided without a service-level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.
Prerequisites
- An Azure Machine Learning workspace. For more information, see Create an Azure Machine Learning workspace.
- The examples repository - Clone the AzureML Example repository. This article uses the assets in
/cli/endpoints/online.
Create a managed online endpoint (preview)
Use the studio to create a managed online endpoint (preview) directly in your browser. When you create a managed online endpoint in the studio, you must define an initial deployment. You cannot create an empty managed online endpoint.
- Go to the Azure Machine Learning studio.
- In the left navigation bar, select the Endpoints page.
- Select + Create (preview).
Follow the setup wizard to configure your managed online endpoint.
- You can use our sample model and scoring script from https://github.com/Azure/azureml-examples/tree/main/cli/endpoints/online/model-1
- On the Environment step of the wizard, you can select the AzureML-sklearn-0.24.1-ubuntu18.04-py37-cpu-inference curated environment.
You can also create a managed online endpoint from the Models page in the studio. This is an easy way to add a model to an existing managed online deployment.
- Go to the Azure Machine Learning studio.
- In the left navigation bar, select the Models page.
- Select a model by checking the circle next to the model name.
- Select Deploy > Deploy to real-time endpoint (preview).
View managed online endpoints (preview)
You can view your managed online endpoints (preview) in the Endpoints page. Use the endpoint details page to find critical information including the endpoint URI, status, testing tools, activity monitors, deployment logs, and sample consumption code:
- In the left navigation bar, select Endpoints.
- (Optional) Create a Filter on Compute type to show only Managed compute types.
- Select an endpoint name to view the endpoint detail page.
Test
Use the Test tab in the endpoints details page to test your managed online deployment. Enter sample input and view the results.
- Select the Test tab in the endpoint's detail page.
- Use the dropdown to select the deployment you want to test.
- Enter sample input.
- Select Test.
Monitoring
Use the Monitoring tab to see high-level activity monitor graphs for your managed online endpoint.
To use the monitoring tab, you must select "Enable Application Insight diagnostic and data collection" when you create your endpoint.
For more information on how viewing additional monitors and alerts, see How to monitor managed online endpoints.
Add a deployment to a managed online endpoint
You can add a deployment to your existing managed online endpoint.
From the Endpoint details page
- Select + Add Deployment button in the endpoint details page.
- Follow the instructions to complete the deployment.
Alternatively, you can use the Models page to add a deployment:
- In the left navigation bar, select the Models page.
- Select a model by checking the circle next to the model name.
- Select Deploy > Deploy to real-time endpoint (preview).
- Choose to deploy to an existing managed online endpoint.
Note
You can adjust the traffic balance between deployments in an endpoint when adding a new deployment.
Update managed online endpoints (preview)
You can update deployment traffic percentage and instance count from Azure Machine Learning studio.
Update deployment traffic allocation
Use deployment traffic allocation to control the percentage of incoming of requests going to each deployment in an endpoint.
- In the endpoint details page, Select Update traffic.
- Adjust your traffic and select Update.
Tip
The Total traffic percentage must sum to either 0% (to disable traffic) or 100% (to enable traffic).
Update deployment instance count
Use the following instructions to scale an individual deployment up or down by adjusting the number of instances:
- In the endpoint details page. Find the card for the deployment you want to update.
- Select the edit icon in the deployment detail card.
- Update the instance count.
- Select Update.
Delete managed online endpoints and deployments (preview)
Learn how to delete an entire managed online endpoint (preview) and it's associated deployments (preview). Or, delete an individual deployment from a managed online endpoint.
Delete a managed online endpoint
Deleting a managed online endpoint also deletes any deployments associated with it.
- Go to the Azure Machine Learning studio.
- In the left navigation bar, select the Endpoints page.
- Select an endpoint by checking the circle next to the model name.
- Select Delete.
Alternatively, you can delete a managed online endpoint directly in the endpoint details page.
Delete an individual deployment
Use the following steps to delete an individual deployment from a managed online endpoint. This does affect the other deployments in the managed online endpoint:
Note
You cannot delete a deployment that has allocated traffic. You must first set traffic allocation for the deployment to 0% before deleting it.
- Go to the Azure Machine Learning studio.
- In the left navigation bar, select the Endpoints page.
- Select your managed online endpoint.
- In the endpoint details page, find the deployment you want to delete.
- Select the delete icon.
Next steps
In this article, you learned how to use Azure Machine Learning managed online endpoints. See these next steps:
- What are endpoints?
- How to deploy managed online endpoints with the Azure CLI
- Deploy models with REST (preview)
- How to monitor managed online endpoints
- Troubleshooting managed online endpoints deployment and scoring (preview)
- View costs for an Azure Machine Learning managed online endpoint (preview)
- Manage and increase quotas for resources with Azure Machine Learning