sakuraime avatar image
0 Votes"
sakuraime asked PRADEEPCHEEKATLA-MSFT commented

Azure synapse spark pool configure authentication to Blob storage

I have a synapse spark spool as well as a blob storage gen 2 .
usually I use Access key and use


to set the access key everytime the notebook run .

  1. how to make it a persisted config of the spark pool , so that I don't need to include spark.conf.set everytime for the spark session ?

  2. apart from Access key , can we use Service Principal or Managed identity to authen to blob inside Synapse Spark ?
    there is tab call credential , and see we reference this credential store from Synapse spark

image.png (3.5 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

0 Votes"

Hello @sakuraime,

To avoid passing access key every time in the notebook run, you can create a linked service to connection to the external data (Azure Blob Storage/Gen1/Gen2).

You can analyze the data in your workspace default ADLS Gen2 account or you can link an ADLS Gen2 or Blob storage account to your workspace through "Manage" > "Linked Services" > "New"


Once a connection is created, the underlying data of that connection will be available for analysis in the Data hub or for pipeline activities in the Integrate hub.


Now you have successfully connected Azure Data Lake Gen2 without passing the access key every time.


Reference: Azure Synapse Analytics - Analyze data in a storage account

Hope this helps. Do let us know if you any further queries.

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

image.png (162.5 KiB)
image.png (70.0 KiB)
image.png (153.0 KiB)
· 8
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I believe in this way you are using AD passthrough authenticate to the blob storage ..

0 Votes 0 ·

Hello @sakuraime,

Apologize for the delay in response.

By default Synapse uses Azure Active Directory (AAD) passthrough by default for authentication between resources. If you need to connect to a resource using other credentials, use the TokenLibrary directly. The TokenLibrary simplifies the process of retrieving SAS tokens, AAD tokens, connection strings, and secrets stored in a linked service or from an Azure Key Vault.

Credentials - helps you to hold authentication details.

Note: . Credentials more like a “shortcut” for TokenLibrary


For more details, refer Secure credentials with linked services using the TokenLibrary.

Hope this helps.

0 Votes 0 ·
image.png (110.8 KiB)
sakuraime avatar image sakuraime PRADEEPCHEEKATLA-MSFT ·

thanks , but how can the spark pool interact with this "Credential" stores?

0 Votes 0 ·
Show more comments