question

lavanyapedada avatar image
0 Votes"
lavanyapedada asked lavanyapedada edited

Azure ML compute not able to access Azure MLS workspace blob( not in vnet) during automl experiment execution

Hi Team,

I'm trying to run the automl code from the examples (https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/automated-machine-learning/regression-explanation-featurization) in Azure MLS which is not in virtual network. While running the experiment, it is getting failed with the below error.

AzureMLCompute job failed.
BFSMountError: Unable to mount blob fuse file system
Info: Could not mount Azure Blob Container azureml-blobstore-xxxx at workspaceblobstore: Unauthorized. Cannot access the storage account with the given account key. Please verify that the account key is valid.
Info: Failed to setup runtime for job execution: Job environment preparation failed on 10.0.0.4 with err exit status 1.

Not sure why the AzureML is not able to access its own blobstorage to place the model artifacts.
The AzureML and the workspace blob both are not in virtual network.

Workarounds tried:
1) Tried to register the workspace blob container (azureml-blobstore-<ID>) as per the link here (https://docs.microsoft.com/en-us/azure/machine-learning/how-to-access-data), but still getting the same error.

Note: The workspace blob storage keys are synced and can able to access the notebooks and data in AzureML, Is this causing the issue?

As per the ticket :- https://docs.microsoft.com/en-us/answers/questions/35043/azure-machine-learning-resync-keys-not-working-no.html

Are the storage keys cached in the storage connection strings at the backend ? however the error message is different, in the reference ticket it says not able to access the resource, but in my case it is not able mount to the azure-ml-<ID> container.

Could you please help on it.

Thanks in advance.




azure-machine-learningazure-blob-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

YutongTie-MSFT avatar image
0 Votes"
YutongTie-MSFT answered

Hello,

Could you please let me know where you try to do that? I tried in my Azure Machine Learning Notebook in the studio and everything works fine for me.

132027-image.png

Please let me know more details and let's see if we can figure this out here. If we need more environment details, I will recommend raising a ticket for this.

Regards,
Yutong


image.png (194.0 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

lavanyapedada avatar image
0 Votes"
lavanyapedada answered lavanyapedada edited

Thanks for your response.

I'm able to connect to workspace and create the compute cluster, I'm getting the error while submitting the experiment on the AML compute.

It is throwing unauthorized error while trying to place the Automl artifacts folder in the AML workspace default blobstore.

132244-error.png





I tired explicitly registering the AML workspace blob container (azureml-blobstore-<ID>) as per the link here (https://docs.microsoft.com/en-us/azure/machine-learning/how-to-access-data), but still getting the same error.

Is it because of resyncing azure blob keys with azure MLS?


error.png (80.6 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.