question

HULLRYAN-3923 avatar image
0 Votes"
HULLRYAN-3923 asked FrankCai-1299 answered

Azure Machine Learning Studios Deploy Model - Obtain logs

I am attempting to deploy a model to an Endpoint in MS Azure Machine Learning studio but I get some errror.

2 Questions
1) How can we get the logs for a deployed model (See my attempts below)
2) How do models get loaded into the docker images as I suspect for some reason it didn't get copied into the docker image.


Step 1) Upload the model: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-existing-model
86639-image.png

Which is successful. I can see and download that my model is correct.

Step 2) Deploy to Azure Container Instances: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-azure-container-instance

I use this code snippet to deploy my code

 from azureml.core.model import InferenceConfig
 from azureml.core.webservice import AciWebservice
 from azureml.core.webservice import Webservice
 from azureml.core.model import Model
 from azureml.core.environment import Environment
    
 script_file_name = 'inference/score.py'
    
 inference_config = InferenceConfig(entry_script=script_file_name, environment=tf_env)
    
 aciconfig = AciWebservice.deploy_configuration(cpu_cores = 1, 
                                                memory_gb = 1, 
                                                tags = {'iris': "rh1832", 'type': "sklearn"}, 
                                                description = 'sample service for iris')
    
 aci_service_name = 'rh1832-iris-demo'
 print(aci_service_name)
 aci_service = Model.deploy(ws, aci_service_name, [model], inference_config, aciconfig)
 aci_service.wait_for_deployment(True)
 print(aci_service.state)

And my score.py inference script looks like

 # ---------------------------------------------------------
 # Copyright (c) Microsoft Corporation. All rights reserved.
 # ---------------------------------------------------------
 import json
 import logging
 import os
 import pickle
 import numpy as np
 import pandas as pd
 import joblib
    
 try:
     logger = logging.getLogger('azureml.automl.core.scoring_script')
 except:
     pass
    
    
 def init():
     global model
     model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_model')
     path = os.path.normpath(model_path)
     path_split = path.split(os.sep)
     logger.info("Loading model from path.")
     model = joblib.load(model_path)
     logger.info("Loading successful.")
    
    
 def run(data):
     try:
         target_names = ['database', 'network', 'resource']
         result = model.predict([data])
         return json.dumps(target_names[result[0]])
     except Exception as e:
         print("error: " + str(e))
         result = str(e)
         return json.dumps({"error": result})


After submitting the webservice to be deployed i get an error that just says its in crashloopbackoff state with no details for which part of the script failed.
86703-image.png

Step 3) I attempted to debug the logs by follow this guide: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-deployment?tabs=azcli#dockerlog

1) Running script locally which I was able to successfully run score.py
2) Running docker image locally I didn't see the model in the folder azureml-environment-setup in the docker container:

root@09e7230839e3:/azureml-environment-setup# ls
environment_context.json log4j.properties mutated_conda_dependencies.yml send_conda_dependencies.py spark_cache.py

and also i didn't see the entrypoint for running the docker container. Is there any guide for running the docker image locally?

3) Finally I attempted to get the logs using Azure CLI and it only return an Null list it appeared
86713-image.png





azure-machine-learningazure-container-instancesazure-machine-learning-inference
image.png (4.1 KiB)
image.png (28.3 KiB)
image.png (5.9 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

FrankCai-1299 avatar image
0 Votes"
FrankCai-1299 answered
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.