question

sakuraime avatar image
0 Votes"
sakuraime asked sakuraime commented

Azure synapse spark Create spark table using and location

In Azure synapse spark pool, I am going to create a spark table using parquet and location

like


%%sql
CREATE TABLE IF NOT EXISTS db.spark_table
USING PARQUET
LOCATION 'wasbs://tables@account.blob.core.windows.net/partitionfile/'


I found I can only authentication with Storage account key ..


HOw can I authenticate using Service principal ?

or can use AD Passthrough authentication?

azure-synapse-analytics
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SamaraSoucy-MSFT avatar image
0 Votes"
SamaraSoucy-MSFT answered sakuraime commented

For Blob Storage Synapse currently only supports SAS authentication. However, you can access the exact same storage from the Data Lake endpoint using pass through authentication. Make sure your Synapse account has the Storage Blob Data Contributor role on the storage account and replace the blob storage location with the data lake one: " abfss://tables@account.dfs.core.windows.net/partitionfile/"

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I think I will need to use Managed private endpoints.

I created a Managed private endpoints. in Synapse workspace to point to my storage account .

And now in synapse spark pool I can use AAD pass through authentication during CREATE TABLE USING PARQUET........using endpoint dfs.core.windows.net .


and While creating this spark table , actually it will automatically create a external source without specifying any credential .

0 Votes 0 ·