question

SunShine-2583 avatar image
0 Votes"
SunShine-2583 asked SunShine-2583 commented

Azure ML Datastore\Datasets

Hello:

I want to know that if it is possible automate copy file from azure storage to Azure ML folder.

I understand that it is duplication of data, but I want to know if yes, how I can do that.

Any pointer is greatly appreciated.

Thanks

azure-machine-learningazure-storage-accounts
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@SunShine-2583 Do you mean copying files from Azure blob storage to your workspace or experiment?
The possible data sources that you can connect to for Azure ML and work with them is documented here.


0 Votes 0 ·

Thanks for your reply.

the link you mentioned will help to access.

I want to see if I can automate the process of copying files to Azure ML Workspace\folder.

Thanks

0 Votes 0 ·

1 Answer

romungi-MSFT avatar image
1 Vote"
romungi-MSFT answered SunShine-2583 commented

Depending on the frequency at which you would like to move data you can create scripts that could run on crontab to move the data between source storage account to your workspace blob store. For example, use azcopy to perform this activity.

A very comprehensive method to move storage between storage accounts is available as a Microsoft learn module that you could take to understand the possibilities and attain this from code to automate in your application.

I would ideally assume that you would like to pull data when your experiment kicks off because you cannot move data to an experiments run id folder unless the experiment has started, In this case you could use the first option to place the data in your workspace blob store and then use it in your experiment without moving it to any other storage. I hope this helps.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

This helps.

Thank you for your help.

0 Votes 0 ·