question

AkashVerma-9570 avatar image
1 Vote"
AkashVerma-9570 asked VoloshinDenis-1121 edited

Azure Data Lake Storage Gen2 Security Features and Backup/Recovery Recommendation

Can someone please let me what security features are available on Azure Data Lake Storage Gen2 and how to plan/implement Backup/Recovery on Adls gen2

azure-data-lake-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @AkashVerma-9570 ,

Thanks your reaching out and using Microsoft Q&A forum.

Security Features
Firstly to talk about Azure Data Lake Storage Gen2 Security features, Data Lake Storage provides six different layers of security: authentication, access control, network isolation, data protection, advanced threat protection, and auditing.

  1. Authentication:
    ADLS Gen2 supports three different authentication methods:

    a) Azure Active Directory is the ideal way to verify a user’s identity. The only potential issue is that users must be defined in azure active directory before they can
    access data.
    b) SAS - Shared Access Signature : You can create a SAS that only has access to specific data and has an expiry date and time, after which it is no longer valid
    c) Shared Access Keys - The caller effectively gains 'super-user' access, meaning full access to all operations on all resources, including setting owner and changing
    ACLs.
    To more about these authentication methods please refer to this doc: Shared Key and Shared Access Signature (SAS) authentication

  2. Access Controls:
    For access control, Azure Data Lake Storage Gen2 implements an access control model that supports both Azure role-based access control (Azure RBAC) and POSIX-like access control lists (ACLs).
    Here is an article which summarizes the basics of the access control model for Data Lake Storage Gen2. Please refer to it: Access control in Azure Data Lake Storage Gen2

  3. Isolation:
    The third layer of security is network isolation. You can actually set up a firewall just for your data lake. Select Firewalls and virtual networks in the Settings menu. The default is to allow access from all networks. If you click Selected networks, then a whole bunch of other configuration options appear.
    i) First, you can enable access from specific virtual networks. Second, you can allow access from particular IP addresses.
    ii) If you want to access your data lake using other Azure services, such as Azure Backup, then you can make an exception by checking this box. Another couple of
    possible exceptions are if you want to allow read access to storage logging and metrics from any network.

  4. Data Protection:
    The fourth layer of security is data protection. ADLS supports encryption of data both at rest and in transit. Data in transit is encrypted using HTTPS by default. Data at rest is also encrypted automatically.

  5. Advanced Security:
    The fifth layer of security is Advanced Threat Protection. If you enable this, it will watch for attempts to access or exploit your storage accounts. If any suspicious activities are detected, then it will send you alerts through Azure Security Center.

  6. Auditing:
    The sixth layer of security is auditing. ADLS logs all account management activities. To see them, click “Activity log”.

For more info please refer to this document: ADLS Gen2 Security recommendations

23769-image.png


Backup/Recovery:
Coming to the second ask in the original query, i.e., regarding Backup/Recovery, as per the latest information from internal sources, ADLS Gen2 Backup integration is in the roadmap but no concrete ETA at the moment. Soft delete is in the current plan and you can expect this feature to be landed in the near future.

I would recommend you to please subscribe to Azure updates to know about the latest updates on Azure products and features.

Here is an existing feature request thread regarding ADLS Gen2 Backup feature, I would encourage you to please comment and/or up-vote as it would help to increase the priority of the feature request : ADLS Gen2 Backup and Point-in-time restore

Additional info : Since ADLS Gen2 doesn't have native support for Backup, as a workaround I would like to share an article/blog (Disclaimer: Not an MSFT document/article) which was found online, please have a look at it and see if the meets your requirement: Custom Backup Azure Data Lake Gen2 using Azure Data Factory

Hope the above information helps.

Thank you


Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.



image.png (26.7 KiB)
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for the response. Security recommendations are fine.
Any other reference or ways for Backup/recovery on ADLS gen2. if any other link contains detailed steps or way ?

0 Votes 0 ·

Hi @AkashVerma-9570,

Unfortunately I wasn't able to find any specific official document that has step by step info related to your query. As per my investigation, ADF is one of the approach. Depending on the data size, Transfer-frequency and network, please prefer any of the approaches described in this document

The following visual illustrates the guidelines to choose the various Azure data transfer tools depending upon the network bandwidth available for transfer, data size intended for transfer, and frequency of the transfer.

24981-image.png


Since there is no out of box feature for Backup/Recovery, I would encourage to up-vote and/or comment on the feature request suggestion shared in my previous response.

Hope this information helps.


Thank you



0 Votes 0 ·
image.png (39.3 KiB)

Continuation to above comment:

You can determine the better suited Azure service for your data transfer from Azure Portal as shown below:

24962-image.png

Hope this helps.



Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.


0 Votes 0 ·
image.png (143.6 KiB)

Hi @AkashVerma-9570,

Just checking in to see if the above information was helpful. If this answers your query, please do click “Mark as Answer” and Up-Vote on the response that helped, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you

0 Votes 0 ·

Hi @AkashVerma-9570,

We still have not heard back from you. Just wanted to check if the above information was helpful? If that answers your query please do click “Mark as Answer” and "Up-Vote" on the response that helped, as it might be beneficial to other community members reading this thread If you have further query, do let us know.

Thank you

0 Votes 0 ·
VoloshinDenis-1121 avatar image
1 Vote"
VoloshinDenis-1121 answered VoloshinDenis-1121 edited

Hi @KranthiPakala-MSFT ,
Is there any update regarding Azure Data Lake Storage Gen2 automatic Backup/Recovery capability, similar to Azure Storage Account blobs?
I saw that there is a feature request that was in the roadmap two years ago.
I'm wondering if there is any progress on this.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.