question

richardwolford-7948 avatar image
0 Votes"
richardwolford-7948 asked MikeUrnun commented

Copy blobs from one storage account to another with a logic app or automation account

I have two storage accounts, one in east us 2 and one in central us. When a blob is added to a container named "test" in east us 2, it needs to be automatically copied to the "test" container in central us. So far I've not been able to accomplish this by way of a logic app.

What's the best way to do this without the need of using a VM and AzCopy for example?

azure-logic-appsazure-storage-accountsazure-blob-storageazure-automation
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @richardwolford-7948 - I feel like that should be a pretty simple setup given the "When a blob is added or modified" trigger and the "Copy Blob" action provided from the Azure Blob Storage connector. Could you elaborate on problems that you have encountered? If possible, please post any error messages and a screenshot of your workflow and be sure to omit any private info.


0 Votes 0 ·

Hi, the problem is that each step I try to use to copy a blob wants the specific name of the blob to copy, which we won't know ahead of time. I need this to be dynamic, whatever the name is of the incoming blob, that's the one that needs copied. "When a blob is added or Modified in Azure Storage" wants the specific blob path.

Using "When a blob is added or modified (properties only)(V2) --> Get Blob Metadata using path (V2) --> Get blob content using path (V2) --> Create blob (V2) I again have the same problem at step 2, it wants the specific blob path, which we won't know ahead of time.

0 Votes 0 ·

Oh and for my four step process, if I hardcode the blob path it runs successfully and copies the named blob. Otherwise it fails.

0 Votes 0 ·
MikeUrnun avatar image MikeUrnun richardwolford-7948 ·

Hi @richardwolford-7948 - Definitely no need to hardcode paths and blob names in the workflow. The data points that you're looking to access (blob properties: name, path etc..) are indeed dynamic during runtime and in fact, referred to as Dynamic Content in Logic Apps workflow. Please review: Perform data operations in Azure Logic Apps doc for more info & context on data operations in Logic Apps.


0 Votes 0 ·

1 Answer

amon-2590 avatar image
0 Votes"
amon-2590 answered MikeUrnun commented

Hi @richardwolford-7948

This can be easily achieved with a combination of Azure Function and azcopy.

The function should be triggered by a blob addition in your storage account, see example here.

Add azcopy as a third party dependency in you Azure function, see example here.
Note: It is recommended to use azcopy to perform the sync because this means you don't need to copy the blob locally, you will simply sync between the storage accounts.

Once a change is detected and the Function is triggered you can copy the file over. Note: You can use the azcopy "sync" option for simplicity for example: .\dependencies\azcopy.exe sync "https://source-storage.blob.core.windows.net/<source-container>?<SAS-Token>" "https://destination-storage.blob.core.windows.net/<destination-container>?<SAS-Token>" --recursive
Extra note1: use the --recursive flag for any nested blobs.
Extra note2: You don't have to use SAS-Token here, using it for simplicity reasons.

Don't forget to add the connection string for the storage account in your App Settings (so that the trigger will work).

That's it! Let us know how it goes :)







· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for that! I had actually thought of this before but there were some issues which came up and I don't remember what they were, this sort've fell by the wayside. But what turned out to work for us was a simple PowerShell script deployed as a runbook. It pulls values out of a vault and uses a service principal for authentication with AAD. But no triggers, so I'm going to give your suggestion a go. My thinking was that since it's PowerShell, other people could work on it without having to learn function apps or logic apps, if that makes sense? I'm still trying out your suggestion, I can report back and see how it goes.

0 Votes 0 ·
MikeUrnun avatar image MikeUrnun richardwolford-7948 ·

Hello @richardwolford-7948 - Just checking in here, any updates based on the AzCopy approach suggested above? Let us know if we can be of any help.

0 Votes 0 ·