Access data in Azure Storage Account Gen2 from Synapse initated by Data Factory with Firewall setup.

Question

question

AxelVulsteke-1308 asked · Mar 05 at 01:23 PM

Access data in Azure Storage Account Gen2 from Synapse initated by Data Factory with Firewall setup.

Hi,

I have a setup as following: Data factory starts a Stored Procedure in Synapse to read data from a delta table like this:

With allowing all networks on the storage account, this works without any problem. But when we only allow selected networks (our VNET + the Synapse and ADF workspace). This fails to work.

The weird thing is:

When I start the stored procedure from within Synapse it works. (I had to setup a private endpoint).
When I want to read data from the storage account immediatly with Data Factory, it works. (I had to setup a private endpoint).

Combining both together do not work.

Is there anywone that can help us with securing our data lake?

Thanks!

azure-data-factory azure-synapse-analytics azure-data-lake-storage

image.png (5.3 KiB)

· 1

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SamaraSoucy-MSFT · Mar 09 at 08:42 AM

Hi Axel,

I am taking a look at your issue. I will reach out if I have any trouble reproducing it.

0 ·

Answer 1 · 2021-03-09T13:38:23.917Z

AxelVulsteke-1308 answered · Mar 09 at 01:38 PM

Hi, in the end we found the issue: we use the Azure Key Vault to provide the credentials towards the Synapse workspace. With this you cannot select "Managed Identity" as an authentication method. Which is needed because we provided the ADF with enough rights on the synapse and underlying storage account.

When you use the "SQL Authentication", that user is used (in our case the admin), that has not enough rights on the storage account.

I hope this also enables others to secure their data lake.

·

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

question