question

92297329 avatar image
0 Votes"
92297329 asked ·

How to configure the ADF pipe line to encrypt and decrpt the data at storage account?

Hi Team,

working on the ADF Pipe line how to encrypt the Data in CSV and following the article https:/_/docs.microsoft.com/en-us/azure/data-factory/enable-customer-managed-key but not encrypting the data of the csv .

if any one know how to do let us know,

thanks in advance

azure-data-factory
· 3
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @SrinivasSrinivas-9999,

Welcome to Microsoft Q&A forum and thanks for your query.

Could you please confirm if you have configured a Customer-Managed Key on an empty data Factory? It is recommended to enable customer-managed key right after factory creation. Also have you set the Soft Delete and Do Not Purge properties on the Key Vault? I would recommend to revisit the Prerequisites one more time to make sure we have followed all the steps.

70164-image.png

Please let us know your observations, so that we can better assist you on this query.

We look forward to your confirmation.

Thanks


0 Votes 0 ·
image.png (13.5 KiB)
92297329 avatar image 92297329 KranthiPakala-MSFT ·

Hi Kranthi,

Thank for the response, yes i have followed the same steps created a customer-managed key on empty data factory only. but after running the pipe line it created the csv without encryption when i opened the csv it showing data as usual. please let us know any other steps we missing it.

thanks in Advance,

0 Votes 0 ·

Hi 92297329,

Thanks for confirming, We are looking into it and will get back to you soon.

Thank you for your patience.

0 Votes 0 ·

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered ·

Hi @92297329 ,

Sorry for the delay in response. I have double checked with internal team about Encrypt Azure Data Factory with customer-managed keys feature and confirmed that this feature is to encrypt the data factory environment i.e., to encrypt data that datafactory storing in the system.

Unfortunately there is no out of box feature in Azure Data factory to perform encryption/decryption of files. Below are couple of feature request suggestions submitted by other Azure users. I would recommend you to please comment and/or up-vote the below user voice items to increase the priority of feature implementation.

https://feedback.azure.com/forums/270578-data-factory/suggestions/40410025-ability-to-decrypt-and-encrypt-the-incoming-and-ou

https://feedback.azure.com/forums/270578-data-factory/suggestions/41023156-add-support-for-gpg-file-decryption

In order to transform/process data in a way that isn't supported by Data Factory, you can create a Custom logic with your own data movement or transformation logic and use the Custom activity in a pipeline. Or another option you could try out is Azure Function Activity.

Additional info :
Here are few related helpful resources:
- https://stackoverflow.com/questions/59684353/azure-pgp-gpg-decryption-of-files
- https://github.com/lfalck/AzureFunctionsPGPDecrypt
- https://docs.microsoft.com/azure/storage/blobs/storage-encrypt-decrypt-blobs-key-vault
- https://medium.com/@sangam.dubey21/azure-data-factory-azure-functions-as-pipeline-activity-part1-cab42f0997c5

Hope this info helps.

Thank you



Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.



· 2 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @92297329 ,

Following up to see if the above information was helpful? If you have further query, please do let us know.

Thank you

0 Votes 0 ·

Hi @92297329 ,

We still have not heard back from you. Just wanted to check if the above info was helpful. Do let us know if you have further query.

Thank you



Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

0 Votes 0 ·