question

Srinivas-9668 avatar image
0 Votes"
Srinivas-9668 asked Srinivas-9668 answered

I have a scenaio to move data from ADLS Gen2 to SQL Server hosted in Azure VM and vice versa.Which ADF - Integration runtime to be used?

How to choose Integration Runtime to load data from ADLSGEN2 to sql server (in Azure VM) and vice versa.
How to decide on number of nodes where IR should be installed?

azure-data-factoryazure-virtual-machines
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PierreLucGiguere-5297 avatar image
0 Votes"
PierreLucGiguere-5297 answered PierreLucGiguere-5297 commented

Hi Srinivas,

Welcome to the community.

You can determine which type of IR via this guide:

https://docs.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime#determining-which-ir-to-use

Sizing is not easy to come up with and I couldn't find any white paper that could help you out. (Note that I do not work for Microsoft so I do have access to only the same resources that you have.

I would personally recommend you perform performance tests with various configurations to better find what would represent an equilibrium between expense and performance for your organization.

Here's a "guide":

https://docs.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@PierreLucGiguere-5297 : Thanks for details. Checked these articles and couldn't use Azure IR to copy /move data from storage account to SQL Server which is hosted in Azure VM.
Does Self- hosted IR is only option for such scenario?

0 Votes 0 ·

Hi Srinivas,

You can do it via both Self-Hosted and Azure IR.

Source: https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-overview#supported-data-stores-and-formats

Since I do not have a clear vision over your infrastructure; I can't really give you all of the big steps I would take but I would definitely go through the following 2 articles to elaborate my test plan:

https://docs.microsoft.com/en-us/azure/data-factory/connector-sql-server
https://docs.microsoft.com/en-us/azure/data-factory/tutorial-hybrid-copy-powershell

0 Votes 0 ·
Srinivas-9668 avatar image
0 Votes"
Srinivas-9668 answered

@PierreLucGiguere-5297 : So can we summarize like this?
If my SQL Server is in public network hosted in Azure VM then I can use Azure IR for data movement.
If SQL Server is in private network hosted in Azure VM then need to use Self-Hosted IR for data movement?

Please suggest.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.