When model is deployed on Kubernetes with model data collection set to true, the dataset which gets registered in workspace has a wrong path.
The blob storage path where this data being collected has default in it but the path registered doesn't have default.
For example, actual path on storage:
azuremlwsdev/modeldata/GUID/ad/ad-workspace-dev/dev-ad-model-v1-2-jen/model/default/inputs
Path dataset is registered with:
GUID/ad/ad-workspace-dev/dev-ad-model-v1-2-jen/model/inputs/**/inputs*.csv
Code inside our scoring service to init collector and use them:
def init():
global inputs_dc, prediction_dc
...
inputs_dc = ModelDataCollector('model', designation="inputs", feature_names=feature_set.columns.values.tolist())
prediction_dc = ModelDataCollector('model', designation="predictions", feature_names=["score"])
def run(raw_data):
inputs_dc.collect(X)
prediction_dc.collect(y_pred)
Below are the library versions being used:
azureml-defaults==1.1.5
azureml-monitoring==0.1.0a21