Use Computer Vision container with Kubernetes and Helm

One option to manage your Computer Vision containers on-premises is to use Kubernetes and Helm. Using Kubernetes and Helm to define a Computer Vision container image, we'll create a Kubernetes package. This package will be deployed to a Kubernetes cluster on-premises. Finally, we'll explore how to test the deployed services. For more information about running Docker containers without Kubernetes orchestration, see install and run Computer Vision containers.

Prerequisites

The following prerequisites before using Computer Vision containers on-premises:

Required Purpose
Azure Account If you don't have an Azure subscription, create a free account before you begin.
Kubernetes CLI The Kubernetes CLI is required for managing the shared credentials from the container registry. Kubernetes is also needed before Helm, which is the Kubernetes package manager.
Helm CLI As part of the Helm CLI install, you'll also need to initialize Helm, which will install Tiller.
Computer Vision resource In order to use the container, you must have:

An Azure Computer Vision resource and the associated API key the endpoint URI. Both values are available on the Overview and Keys pages for the resource and are required to start the container.

{API_KEY}: One of the two available resource keys on the Keys page

{ENDPOINT_URI}: The endpoint as provided on the Overview page

Gathering required parameters

There are three primary parameters for all Cognitive Services' containers that are required. The end-user license agreement (EULA) must be present with a value of accept. Additionally, both an Endpoint URL and API Key are needed.

Endpoint URI {ENDPOINT_URI}

The Endpoint URI value is available on the Azure portal Overview page of the corresponding Cognitive Service resource. Navigate to the Overview page, hover over the Endpoint, and a Copy to clipboard icon will appear. Copy and use where needed.

Gather the endpoint uri for later use

Keys {API_KEY}

This key is used to start the container, and is available on the Azure portal's Keys page of the corresponding Cognitive Service resource. Navigate to the Keys page, and click on the Copy to clipboard icon.

Get one of the two keys for later use

Important

These subscription keys are used to access your Cognitive Service API. Do not share your keys. Store them securely, for example, using Azure Key Vault. We also recommend regenerating these keys regularly. Only one key is necessary to make an API call. When regenerating the first key, you can use the second key for continued access to the service.

The host computer

The host is a x64-based computer that runs the Docker container. It can be a computer on your premises or a Docker hosting service in Azure, such as:

Container requirements and recommendations

Note

The requirements and recommendations are based on benchmarks with a single request per second, using an 8-MB image of a scanned business letter that contains 29 lines and a total of 803 characters.

The following table describes the minimum and recommended allocation of resources for each Read container.

Container Minimum Recommended TPS
(Minimum, Maximum)
Read 1 cores, 8-GB memory, 0.24 TPS 8 cores, 16-GB memory, 1.17 TPS 0.24, 1.17
  • Each core must be at least 2.6 gigahertz (GHz) or faster.
  • TPS - transactions per second.

Core and memory correspond to the --cpus and --memory settings, which are used as part of the docker run command.

Connect to the Kubernetes cluster

The host computer is expected to have an available Kubernetes cluster. See this tutorial on deploying a Kubernetes cluster for a conceptual understanding of how to deploy a Kubernetes cluster to a host computer.

Sharing Docker credentials with the Kubernetes cluster

To allow the Kubernetes cluster to docker pull the configured image(s) from the containerpreview.azurecr.io container registry, you need to transfer the docker credentials into the cluster. Execute the kubectl create command below to create a docker-registry secret based on the credentials provided from the container registry access prerequisite.

From your command-line interface of choice, run the following command. Be sure to replace the <username>, <password>, and <email-address> with the container registry credentials.

kubectl create secret docker-registry containerpreview \
    --docker-server=containerpreview.azurecr.io \
    --docker-username=<username> \
    --docker-password=<password> \
    --docker-email=<email-address>

Note

If you already have access to the containerpreview.azurecr.io container registry, you could create a Kubernetes secret using the generic flag instead. Consider the following command that executes against your Docker configuration JSON.

 kubectl create secret generic containerpreview \
     --from-file=.dockerconfigjson=~/.docker/config.json \
     --type=kubernetes.io/dockerconfigjson

The following output is printed to the console when the secret has been successfully created.

secret "containerpreview" created

To verify that the secret has been created, execute the kubectl get with the secrets flag.

kubectl get secrets

Executing the kubectl get secrets prints all the configured secrets.

NAME                  TYPE                                  DATA      AGE
containerpreview      kubernetes.io/dockerconfigjson        1         30s

Configure Helm chart values for deployment

Start by creating a folder named read, then paste the following YAML content into a new file named Chart.yml.

apiVersion: v1
name: read
version: 1.0.0
description: A Helm chart to deploy the microsoft/cognitive-services-read to a Kubernetes cluster

To configure the Helm chart default values, copy and paste the following YAML into a file named values.yaml. Replace the # {ENDPOINT_URI} and # {API_KEY} comments with your own values.

# These settings are deployment specific and users can provide customizations

read:
  enabled: true
  image:
    name: cognitive-services-read
    registry: containerpreview.azurecr.io/
    repository: microsoft/cognitive-services-read
    tag: latest
    pullSecret: containerpreview # Or an existing secret
    args:
      eula: accept
      billing: # {ENDPOINT_URI}
      apikey: # {API_KEY}

Important

If the billing and apikey values are not provided, the services will expire after 15 min. Likewise, verification will fail as the services will not be available.

Create a templates folder under the read directory. Copy and paste the following YAML into a file named deployment.yaml. The deployment.yaml file will serve as a Helm template.

Templates generate manifest files, which are YAML-formatted resource descriptions that Kubernetes can understand. - Helm Chart Template Guide

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: read
spec:
  template:
    metadata:
      labels:
        app: read-app
    spec:
      containers:
      - name: {{.Values.read.image.name}}
        image: {{.Values.read.image.registry}}{{.Values.read.image.repository}}
        ports:
        - containerPort: 5000
        env:
        - name: EULA
          value: {{.Values.read.image.args.eula}}
        - name: billing
          value: {{.Values.read.image.args.billing}}
        - name: apikey
          value: {{.Values.read.image.args.apikey}}
      imagePullSecrets:
      - name: {{.Values.read.image.pullSecret}}

--- 
apiVersion: v1
kind: Service
metadata:
  name: read
spec:
  type: LoadBalancer
  ports:
  - port: 5000
  selector:
    app: read-app

The template specifies a load balancer service and the deployment of your container/image for Read.

The Kubernetes package (Helm chart)

The Helm chart contains the configuration of which docker image(s) to pull from the containerpreview.azurecr.io container registry.

A Helm chart is a collection of files that describe a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on.

The provided Helm charts pull the docker images of the Computer Vision Service, and the corresponding service from the containerpreview.azurecr.io container registry.

Install the Helm chart on the Kubernetes cluster

To install the helm chart, we'll need to execute the helm install command. Ensure to execute the install command from the directory above the read folder.

helm install read --name read

Here is an example output you might expect to see from a successful install execution:

NAME: read
LAST DEPLOYED: Thu Sep 04 13:24:06 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME                    READY  STATUS             RESTARTS  AGE
read-57cb76bcf7-45sdh   0/1    ContainerCreating  0         0s

==> v1/Service
NAME     TYPE          CLUSTER-IP    EXTERNAL-IP  PORT(S)         AGE
read     LoadBalancer  10.110.44.86  localhost    5000:31301/TCP  0s

==> v1beta1/Deployment
NAME    READY  UP-TO-DATE  AVAILABLE  AGE
read    0/1    1           0          0s

The Kubernetes deployment can take over several minutes to complete. To confirm that both pods and services are properly deployed and available, execute the following command:

kubectl get all

You should expect to see something similar to the following output:

kubectl get all
NAME                        READY   STATUS    RESTARTS   AGE
pod/read-57cb76bcf7-45sdh   1/1     Running   0          17s

NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
service/kubernetes     ClusterIP      10.96.0.1      <none>        443/TCP          45h
service/read           LoadBalancer   10.110.44.86   localhost     5000:31301/TCP   17s

NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/read   1/1     1            1           17s

NAME                              DESIRED   CURRENT   READY   AGE
replicaset.apps/read-57cb76bcf7   1         1         1       17s

Validate that a container is running

There are several ways to validate that the container is running. Locate the External IP address and exposed port of the container in question, and open your favorite web browser. Use the various request URLs below to validate the container is running. The example request URLs listed below are http://localhost:5000, but your specific container may vary. Keep in mind that you're to rely on your container's External IP address and exposed port.

Request URL Purpose
http://localhost:5000/ The container provides a home page.
http://localhost:5000/status Requested with an HTTP GET, to validate that the container is running without causing an endpoint query. This request can be used for Kubernetes liveness and readiness probes.
http://localhost:5000/swagger The container provides a full set of documentation for the endpoints and a Try it out feature. With this feature, you can enter your settings into a web-based HTML form and make the query without having to write any code. After the query returns, an example CURL command is provided to demonstrate the HTTP headers and body format that's required.

Container's home page

Next steps

For more details on installing applications with Helm in Azure Kubernetes Service (AKS), visit here.