question

ValentinDuhamel-5768 avatar image
0 Votes"
ValentinDuhamel-5768 asked PRADEEPCHEEKATLA-MSFT commented

Connect to storage account using private endpoint from a Notebook attached to a spark pool in Azure Synapse Analytics in a custom vNet

Hello

We are trying to connect to a storage account using private endpoint from a Notebook attached to a spark pool on Synapse.

Here are some key points of our configuration:

  • We are NOT using the managed VNET and managed private endpoints features from Synapse. Our client requirements is to use a custom vNet.

  • We created a private endpoint for the subresource "dfs" of the storage account, and configured the private DNS zone accordingly.

  • We also create private endpoints for subresources "Dev" and "SqlOnDemand" for Synapse (we are not using dedicated pool), private DNS zones and in the same vNet too.

  • We checked the DNS resolution, it's resolving the name privately.

  • It's working when the storage account is set to : "Enabled from all networks"

  • All components (synapse workspace, storage account, vnet are on the same subscription, same resource group and same location)

Below a diagram on what we are trying to achieve:
222606-image.png

When we set the storage account to : "Enabled from selected virtual networks and IP adresses" with the configuration below:

222683-image.png

We get this error message: Caused by: Operation failed: "This request is not authorized to perform this operation.", 403, GET,

So the question behind is: is it possible to reach a storage account from a spark pool using custom private endpoints in a custom vNET?


azure-synapse-analyticsazure-virtual-networkazure-data-lake-storagedotnet-ml-big-data
image.png (62.5 KiB)
image.png (42.4 KiB)
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

ValentinDuhamel-5768 avatar image
1 Vote"
ValentinDuhamel-5768 answered PRADEEPCHEEKATLA-MSFT commented

I had an answer from Microsoft Support, it's not possible to connect to an Azure Data Lake store account with firewall enabled in a Non-managed VNET and they're not planned to support this.

· 1
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT avatar image PRADEEPCHEEKATLA-MSFT ·

Hello @ValentinDuhamel-5768,

Glad to know that you have already got answer from support team. And thanks for sharing the solution, which might be beneficial to other community members reading this thread.

0 Votes 0 ·