question

GabrielHabre-0163 avatar image
0 Votes"
GabrielHabre-0163 asked GabrielHabre-0163 commented

Azure Storage Tables Backup Options

After some searching, it seems that even today there aren't many options provided by Azure to back up the azure storage tables.
We have managed to use the older version of AzCopy 7.1, which requires reducing TLS Security on the storage account first to 1.0 every time we want to do a backup. Additionally, this doesn't just pick up all tables and needs a script to be maintained every time a new table is added.
We can probably also build our own .NET code that just reads from one storage account and dumps into another, however this way we lose the Timestamp column when restoring, which can be important to track logs and issues.
Are Azure Storage Tables being discontinued?
Why can't we seem to find any Azure Cloud-based backup offering? Perhaps we should resort to third-party providers or code our own.
Thanks

azure-table-storage
· 5
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

OuryBa-MSFT avatar image OuryBa-MSFT ·

Hi @GabrielHabre-0163 Welcome to Microsoft Q&A and thank you for posting your Question.

I understand your pain when trying to back up Azure tables storage. We are working with our Product group to understand more about this behavior. We will reach out to you at the earliest.

Regards,
Oury

1 Vote 1 ·
jamieg avatar image jamieg OuryBa-MSFT ·

This would be a great feature, certainly now the azcopy does not support Table storage anymore.

Kind regards,
Jamie

1 Vote 1 ·
MaartenKieft-1010 avatar image MaartenKieft-1010 OuryBa-MSFT ·

Yes I'm really looking for something which can help me with this.

1 Vote 1 ·
OuryBa-MSFT avatar image OuryBa-MSFT ·

@GabrielHabre-0163
Sorry for the delay in my response to your question.
After checking internally with the team. Looks like Azure Table Storage doesn't have any backup capability. The closest you'll get is Storage's geo-redundancy but that's more a DR feature and won't protect you from user/client errors.

Are Azure Storage Tables being discontinued?
No Aure tables storage will not be discontinued

Regards,
Oury



0 Votes 0 ·
AlejandrodelaFuenteydelaRosa avatar image AlejandrodelaFuenteydelaRosa ·

I was using a script with azcopy73. But it only works on windows and has a huge disadvantage. The one that to make the copy you first have to download it, pass it to a blobs and then insert it in the destination table... If you have a large volume of data (as in my case) it will stop being useful.
A solution can be to have another storage where you create the same tables and make the queries to the origin and inserts to a table of the new storage. Having in some of these a table of the date of the last backup, so as not to consult elements already copied again.
(obviously being useless in the case of synchronizing deleted entities)

I'm still looking for a truly useful solution for this task. Since that's a ridiculous workload for functionality that should be built in...

I hope it will help you and if you find a better solution, do not hesitate to communicate it.

0 Votes 0 ·

2 Answers

JurjenLadeniusEffectory-0890 avatar image
1 Vote"
JurjenLadeniusEffectory-0890 answered GabrielHabre-0163 commented

You can actually use Az-Copy 7.3 that does support TLS1.2.
We automated something using powershell and AZ-Copy 7.3. For us it also stops being useful because of volume.

But since the current version is v10 and MS actually tells us that Table Storage will never be supported by never versions of Az-Copy, there really is no way to backup Table Storage:
https://github.com/Azure/azure-storage-azcopy/issues/1420

ADF seems to be able to copy data, but I'm not so sure about maintainability and cost: https://techcommunity.microsoft.com/t5/azure-paas-blog/copying-azure-table-storage-using-azure-data-factory-adf/ba-p/3062401
When making backups I'm not interested in replication, creating e.g. daily/weekly/monthly backups is important. You can achieve replication with geo-redundancy, ADF just gives a window because of scheduling.

Then MS seems to see Cosmos DB as the replacement for table storage. I can see why: It is way more lucrative.

· 4
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GabrielHabre-0163 avatar image GabrielHabre-0163 ·

Hi Jurien,

You are right, Az-Copy 7.3 does support TLS1.2, but gives issues if you don't do some registry changes first on windows:

SystemDefaultTlsVersions & SchUseStrongCrypto should be added as described in below links:

https://github.com/microsoft/azure-pipelines-tasks/issues/10337
https://docs.microsoft.com/en-us/mem/configmgr/core/plan-design/security/enable-tls-1-2-client

We will be building our own solution after all, which will be backing up to azure blob storage and then using azure backup for the blobs

0 Votes 0 ·
JurjenLadeniusEffectory-0890 avatar image JurjenLadeniusEffectory-0890 GabrielHabre-0163 ·

All our Azure storage accounts have been configured to accept HTTPS only with a minimum TLS version of 1.2 using Azure Policy.
199058-image.png

We are running table storage backups using Az-Copy in build pipelines, using powershell and az-copy to create timestamped copies in blob storage.
We haven't had any issues or need for configuration. (though I can assure you that any connection less than TLS1.2 will fail)

Using a really old unmaintained Az-Copy does pose a risk for us as it may stop function at some time or introduce a vulnerability
So the only other option would be a custom solution as there really isn't any other 3rd party solution.


1 Vote 1 ·
image.png (7.8 KiB)
JurjenLadeniusEffectory-0890 avatar image JurjenLadeniusEffectory-0890 JurjenLadeniusEffectory-0890 ·

I do see why a host may need to be configured for TLS1.2 with .NET 4.x , we just didn't need any on the hosted build agents.

1 Vote 1 ·
Show more comments
Wafflehouse-7201 avatar image
1 Vote"
Wafflehouse-7201 answered

You can use Azure Data Factory and setup pipelines to copy the Storage table to one or more additional storage accounts, more of a snapshot than a backup, but a way to create additional copies.

5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.