question

SrinivasTallam-4011 avatar image
0 Votes"
SrinivasTallam-4011 asked ShaikMaheer-MSFT commented

ADF Architecture (or) how does pipeline executed in SHIR?

Hi,

Please let us know how does the ADF pipeline execute in the VM or AZURE IR?

Does the pipeline converted into scala or java or .net program before submitting to IR?
What is the ADF Architecture?
What is the backbone of ADF?


Regards,

azure-data-factory
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hey,
Just to clarify , is your ask about how to trigger an ADF pipeline from some other method other than in built triggers ?

0 Votes 0 ·

Does the pipeline converted into scala or java or .net program before submitting to IR?

Is it cluster based execution or memory based execution?

0 Votes 0 ·

Hi @SrinivasTallam-4011 ,

Following up to see if below answer helps you? If yes, Please Accept Answer. Accepted answers helps community as well. Please let us know if any further queries. Thank you.

0 Votes 0 ·

Hi @SrinivasTallam-4011 ,

Just checking if below answer helps you? If yes, Please Accept Answer. Accepted answers helps community as well. Please let us know if any further queries. Thank you.

0 Votes 0 ·

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered

Hi @SrinivasTallam-4011 ,

Thank you for posting query in Microsoft Q&A Platform.

Azure Data Factory is a managed cloud service that's built for complex hybrid extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects.

The Integration Runtime (IR) is the compute infrastructure used by Azure Data Factory to provide the following data integration capabilities across different network environments:

  • Data Flow

  • Data Movement

  • Activity Dispatch

  • SSIS Package Execution

Integration runtime acts as bridge between Activity(Task which user wants to perform) and linked service(Target resource).

For example, if your task is to run SQL script using lookup activity, then IR will help ADF to connect with SQL server(linked service) and execute sql query on it. So, here your SQL server which is configured on linked service acts as compute environment. Now Query execution mechanism is purely depending your compute resource type.

So, ADF don't have its own compute environment, depending on resource your are connecting(linked service) to execute activity, it utilizes that resource compute to complete task.

Hope this will help. Please let us know if any further queries. Thank you.


  • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.