question

RaGus-6891 avatar image
0 Votes"
RaGus-6891 asked MuhammadHamza-4653 answered

Synapse Analytics - Can't connect to external storage account

Dear All,

I am trying to connect a (new) Synapse Analytics workspace to an external storage account (datalake gen2) using a py spark notebook (using ddedicated spark pool).
In order to read with a simple query construct I have added my current account in use (userXXX) as Storage Blob Data Contributor to the container1 (using IAM).

When running the following command in a cell

 account_name = 'mystorageaccountname'
 container_name = 'container1'
 relative_path = 'path_to_parquet_files'
 adls_path = 'abfss://%s@%s.dfs.core.windows.net/%s' % (container_name, account_name, relative_path)
    
 df = spark.read.parquet(adls_path)
 df.show()

I receive the error

Py4JJavaError : An error occurred while calling o334.parquet.
: Operation failed: "This request is not authorized to perform this operation.", 403, HEAD, https://mystorageaccountname.dfs.core.windows.net/container1//?upn=false&action=getAccessControl&timeout=90

Do you have suggestion in order to solve this issue? Is there something I have forgot?

Thanks
Ra

azure-synapse-analytics
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MuhammadHamza-4653 avatar image
0 Votes"
MuhammadHamza-4653 answered

Hi,
I tried something and it worked...!
The notebook runs with the current user in synapse studio as its service principle. We need to give extra permission to that 'user'. The permission is "Storage Blob Data Contributor" or "Storage Blob Data Owner".

For example,
If the user who is running the notebook from synapse studio currently has "Contributor" or "Owner" rights at the storage account level it will not be going to work however, we have to give that user one extra permission, The permission is "Storage Blob Data Contributor" or "Storage Blob Data Owner".

Please let me know if it works for you

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KowsiMurugapoopathy-6155 avatar image
0 Votes"
KowsiMurugapoopathy-6155 answered TarapareddySaikrishna-4046 commented

Hey @euangMS,

I'm getting the same issue as well.

Py4JJavaError : An error occurred while calling o416.load.
: Operation failed: "Server failed to authenticate the request. Please refer to the information in the www-authenticate header.", 401, HEAD,

I tried authenticating using Managed Key and Managed Identity still no-bueno



· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @KowsiMurugapoopathy-6155 @RaGus-6891

any luck..?? were u able to access external storage account from synapse workspace.

i have grated the workspace identity as storage blob contributor and still getting the same error.

Regards,
Sai

0 Votes 0 ·
euangMS avatar image
0 Votes"
euangMS answered

Sorry I don't have anything else in mind, the error is pretty sefl explanatory, you don't have permissions (although the location not existing will also throw the same error)>

When you create a Synapse Workspace we ask for a default storage account and we set all the right permissions for the user who is setting up the WS, but for other users they need to be granted Storage Blob Data Contributer rights but thats not whats needed to just read files.

In general I would say this is permissions or existence problem.

-Euan

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RaGus-6891 avatar image
0 Votes"
RaGus-6891 answered

HI Euan,

yes I have already attache the same storage account as linked service. As far as I know linked service is needed by the pipeline "section".

Do you have suggestions?

Thanks
Ra

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

euangMS avatar image
0 Votes"
euangMS answered

Did you add the storage account as a linked service? Can you browse it in the linked service/storage UI?

-Euan

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.