Copy blobs from one storage account to another with a logic app or automation account

Richard Duane Wolford Jr 206

I have two storage accounts, one in east us 2 and one in central us. When a blob is added to a container named "test" in east us 2, it needs to be automatically copied to the "test" container in central us. So far I've not been able to accomplish this by way of a logic app.

What's the best way to do this without the need of using a VM and AzCopy for example?

Mike Urnun 9,756 Reputation points Microsoft Employee

2021-09-09T21:52:14.77+00:00

Hello @richardwolford-7948 - I feel like that should be a pretty simple setup given the "When a blob is added or modified" trigger and the "Copy Blob" action provided from the Azure Blob Storage connector. Could you elaborate on problems that you have encountered? If possible, please post any error messages and a screenshot of your workflow and be sure to omit any private info.
Richard Duane Wolford Jr 206 Reputation points

2021-09-13T15:43:33.83+00:00

Hi, the problem is that each step I try to use to copy a blob wants the specific name of the blob to copy, which we won't know ahead of time. I need this to be dynamic, whatever the name is of the incoming blob, that's the one that needs copied. "When a blob is added or Modified in Azure Storage" wants the specific blob path.

Using "When a blob is added or modified (properties only)(V2) --> Get Blob Metadata using path (V2) --> Get blob content using path (V2) --> Create blob (V2) I again have the same problem at step 2, it wants the specific blob path, which we won't know ahead of time.
Richard Duane Wolford Jr 206 Reputation points

2021-09-13T15:44:38.65+00:00

Oh and for my four step process, if I hardcode the blob path it runs successfully and copies the named blob. Otherwise it fails.
Mike Urnun 9,756 Reputation points Microsoft Employee

2021-09-20T02:14:12.533+00:00

Hi @richardwolford-7948 - Definitely no need to hardcode paths and blob names in the workflow. The data points that you're looking to access (blob properties: name, path etc..) are indeed dynamic during runtime and in fact, referred to as Dynamic Content in Logic Apps workflow. Please review: Perform data operations in Azure Logic Apps doc for more info & context on data operations in Logic Apps.

1 answer

amon 121 Reputation points Microsoft Employee

2021-09-14T12:48:29.537+00:00

Hi @richardwolford-7948

This can be easily achieved with a combination of Azure Function and azcopy.

The function should be triggered by a blob addition in your storage account, see example here.

Add azcopy as a third party dependency in you Azure function, see example here.
Note: It is recommended to use azcopy to perform the sync because this means you don't need to copy the blob locally, you will simply sync between the storage accounts.

Once a change is detected and the Function is triggered you can copy the file over. Note: You can use the azcopy "sync" option for simplicity for example: .\dependencies\azcopy.exe sync "https://source-storage.blob.core.windows.net/<source-container>?<SAS-Token>" "https://destination-storage.blob.core.windows.net/<destination-container>?<SAS-Token>" --recursive
Extra note1: use the --recursive flag for any nested blobs.
Extra note2: You don't have to use SAS-Token here, using it for simplicity reasons.

Don't forget to add the connection string for the storage account in your App Settings (so that the trigger will work).

That's it! Let us know how it goes :)
Please sign in to rate this answer.

1 person found this answer helpful.
Richard Duane Wolford Jr 206 Reputation points

2021-09-14T17:58:46.543+00:00

Thanks for that! I had actually thought of this before but there were some issues which came up and I don't remember what they were, this sort've fell by the wayside. But what turned out to work for us was a simple PowerShell script deployed as a runbook. It pulls values out of a vault and uses a service principal for authentication with AAD. But no triggers, so I'm going to give your suggestion a go. My thinking was that since it's PowerShell, other people could work on it without having to learn function apps or logic apps, if that makes sense? I'm still trying out your suggestion, I can report back and see how it goes.

Mike Urnun 9,756 Reputation points Microsoft Employee

2021-09-20T02:04:42.337+00:00

Hello @richardwolford-7948 - Just checking in here, any updates based on the AzCopy approach suggested above? Let us know if we can be of any help.
Sign in to comment