question

2JK avatar image
0 Votes"
2JK asked 2JK commented

Loading pickle object in entry script in Azure ML

I have an entry script that loads a pickled tokenizer object from Tensorflow and the model itself. When I try to deploy, locally or otherwise, I get an error saying something broke in the init function in the score.py script. Commenting out the tokenizer and the deployment works so I'm sure it's because of it. This is how I define the function:

 def init():
     global tokenizer, model
     tokenizer_path = os.path.join('./objs', 'tokenizer.pkl') # tried absolute path as well, didn't work
     tokenizer = pickle.load(tokenizer_path)
     # tokenizer = pickle.load(open(tokenizer_path, 'rb')) # also tried this, didn't work
     model = tf.keras.models.load_model(os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'model.h5'))

Is that the correct way to load a pickle object in the entry script? Any tips would be appreciated.

azure-machine-learning
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

romungi-MSFT avatar image
0 Votes"
romungi-MSFT answered 2JK commented

@2JK I think this should help.

from sklearn.externals import joblib

 tokenizer_path = os.path.join('./objs', 'tokenizer.pkl') # tried absolute path as well, didn't work
 tokenizer = joblib.load(tokenizer_path)


Did you also try the absolute path in your tokenizer_path?

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you for the answer.

Unfortunately, this did not work as well. I had to upload the pickle object in a container, connect to the blob client, download the pickle blob, then load it using pickle.loads() to make it work. Not sure if I missed something using the other way.

1 Vote 1 ·