Has anyone tried a linked service in Azure SQL MI connected to Synapse Serverless as per the approach in this article? What does performance look like for importing millions of rows from parquet files in ADLS?
Has anyone tried a linked service in Azure SQL MI connected to Synapse Serverless as per the approach in this article? What does performance look like for importing millions of rows from parquet files in ADLS?
Hi @TeoLachev-9086 Thank you for posting your question on Microsoft Q&A.
What does performance look like for importing millions of rows from parquet files in ADLS?
We would expect the performance to be affected by the Synapse tier and the network bandwidth, plus the insertion that probably will be single-threaded, but which can be solved through multiple parallel processes.
Instead of starting one process inserting all the data, split it into multiple processes (SQL Agent Jobs, for example) for potentially achieving better insertion speed. This assumes the abundance of the resources such as Network Bandwidth, Synapse processing power and of course the source file system itself, etc)
Regards,
Oury
Hi @TeoLachev-9086 Checking if the above answer was helpful, If yes please accept as answer otherwise let us know how we can better help you.
Regards,
Oury
Hi Qory,
It was helpful, thank you. Based on my internal tests, querying a Serverless external table via MI is around x10 slower that using OPENROWSET, which I contribute to performance differences between Polybase and Serverless.
10 people are following this question.