question

thracian avatar image
0 Votes"
thracian asked thracian answered

Performance Feedback for Azure SQL MI Linked Service to Synapse Serverless

Has anyone tried a linked service in Azure SQL MI connected to Synapse Serverless as per the approach in this article? What does performance look like for importing millions of rows from parquet files in ADLS?




azure-sql-database
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

OuryBa-MSFT avatar image
1 Vote"
OuryBa-MSFT answered OuryBa-MSFT commented

Hi @TeoLachev-9086 Thank you for posting your question on Microsoft Q&A.
What does performance look like for importing millions of rows from parquet files in ADLS?
We would expect the performance to be affected by the Synapse tier and the network bandwidth, plus the insertion that probably will be single-threaded, but which can be solved through multiple parallel processes.
Instead of starting one process inserting all the data, split it into multiple processes (SQL Agent Jobs, for example) for potentially achieving better insertion speed. This assumes the abundance of the resources such as Network Bandwidth, Synapse processing power and of course the source file system itself, etc)

Regards,
Oury

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @TeoLachev-9086 Checking if the above answer was helpful, If yes please accept as answer otherwise let us know how we can better help you.

Regards,
Oury

0 Votes 0 ·
thracian avatar image
0 Votes"
thracian answered

Hi Qory,

It was helpful, thank you. Based on my internal tests, querying a Serverless external table via MI is around x10 slower that using OPENROWSET, which I contribute to performance differences between Polybase and Serverless.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.