Synapse can't access storage (403)

Question

question

LHTDKI-5572 asked Apr 15, '22 GitaraniSharmaMSFT-4262 edited May 9, '22

Synapse can't access storage (403)

Hello everyone,

I am facing a firewall issue that prevents my pipeline from working correctly :

The pipeline (Azure Synapse) is supposed to be triggered when files are deposited in a blob storage directory, copying them from 'ENTREE' to 'EN_COURS'.

Everything was working fine when the firewall was set to 'All Networks'

Here are the IP and Azure Instances allowed in the networking :

After setting up an ip whitelisting so only us and our clients could access the portal I am facing this error after the pipeline is triggered by a file being deposited in the blob directory (ENTREE) :

"ErrorCode=AzureBlobOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Blob operation Failed. ContainerName: https://dkdatalakecanon.blob.core.windows.net/dk-storage-canon-medical, path: ENTREE/PRODUITS/UL_Resah_2019_070_LOT20_A000_AV5.xlsx.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The remote server returned an error: (403) Forbidden.,Source=,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=RequestId:2c9299fd-201e-001d-49fa-4f8d18000000 Time:Thu, 14 Apr 2022 12:21:54 GMT,,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,'", "failureType": "UserError", "target": "

After adding the IPs found here (https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-configure-firewall), I have another error :

{ "errorCode": "2200", "message": "ErrorCode=AzureBlobOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Blob operation Failed. ContainerName: https://dkdatalakecanon.blob.core.windows.net/dk-storage-canon-medical, path: ENTREE/PRODUITS/UL_Resah_2019-070_LOT20_A000_AV5.xlsx.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The remote server returned an error: (403) Forbidden.,Source=,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=RequestId:276787cf-601e-006e-48d2-50d58b000000 Time:Fri, 15 Apr 2022 14:06:57 GMT,,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,'", "failureType": "UserError", "target": "Copie_Entree_To_En_Cours", "details": [] }

Thanks in advance for your help !

azure-synapse-analytics azure-blob-storage

image.png (8.1 KiB)

image.png (19.9 KiB)

image.png (11.4 KiB)

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT · Apr 19 at 07:03 AM

Hello @LHTDKI-5572,

Thanks for the question and using MS Q&A platform.

When you say - After setting up an ip whitelisting so only us and our clients could access the portal I am facing this error, could you please confirm whether your IP address whitelisted?

0 ·

Answer 1 · 2022-04-19T07:10:32.497Z

LHTDKI-5572 answered Apr 19, '22 LHTDKI-5572 commented Apr 22, '22

Hello @PRADEEPCHEEKATLA-MSFT,

My IP address is whitelisted as I can access the sftp server that deposit the file into the blob.

· 2

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT · Apr 22 at 05:34 AM

Hello @LHTDKI-5572,

I agree that this issue looks strange and I wasn't able to reproduce this issue. If you have a support plan could you please file a support ticket for deeper investigation.

1 ·

LHTDKI-5572 PRADEEPCHEEKATLA-MSFT · Apr 22 at 07:38 AM

Thank you for your time.
I cannot file a ticket due to my plan being pay per use, I'll look into creating virtual networks to try to solve this problem.

0 ·

Answer 2 · 2022-04-22T08:33:17.227Z

AzureAaronHughes answered Apr 22, '22

Try to create a private endpoint to the storage account to access from Synapse work space - this will use the backbone rather than public net

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

question