question

56220211 avatar image
0 Votes"
56220211 asked PRADEEPCHEEKATLA-MSFT edited

Time-to-live in Synapse IR

I see that the TTL is now officially available in Synapse. Few questions:

https://docs.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime-performance#time-to-live

1) The documentation says that "it is not available when using the auto-resolve integration runtime" even though I could see the option under the auto-resolve IR as well. What does this really mean?

134106-image.png

2) Will this help even if I execute the same data flow multiple times sequentially inside a looping activity?


3) If we run multiple pipelines in parallel and if each pipeline has multiple data flows executed sequentially inside, are there any risk in enabling TTL?

azure-synapse-analytics
image.png (18.4 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I1) I have Time To Live turned on in my custom integration runtimes, and the Region is set to "Auto Resolve", the Region actually needs to be set to "Auto Resolve".

The Auto Resolve Integration Runtime that does not support Time To Live is the default integration runtime that is created for you.

134578-image.png



2) Yes the loop will start your custom integration runtime on the first time through the loop, and subsequent iterations should use the custom integration runtime.

3) It should start a new custom integration runtime for each parallel pipeline

0 Votes 0 ·
image.png (2.3 KiB)

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT edited

Hello @56220211,


[UPDATE: 10/6/2021] This is confirmed as the product bug and fixed by the ADF team.



Thanks for the question and using MS Q&A platform.

1) The documentation says that "it is not available when using the auto-resolve integration runtime" even though I could see the option under the auto-resolve IR as well. What does this really mean?

When you say you are able to see "Time to Live" options, are you able to see Apply button?

As per my repro, I see this feature TTL is not available when using the auto-resolve integration runtime.

2) Will this help even if I execute the same data flow multiple times sequentially inside a looping activity?

TTL will help to not stop cluster so that immediate next dataflow can utilize it and not require to wait for new cluster up. This will reduce dataflow over all execution time.

3) If we run multiple pipelines in parallel and if each pipeline has multiple data flows executed sequentially inside, are there any risk in enabling TTL?

No there is no risk. TTL will only help to avoid cluster spin time.

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Yes, I see the Apply button under the auto-resolve runtime and we set it to 5 mins and able to save.

134472-image.png


0 Votes 0 ·
image.png (41.2 KiB)

Hello @56220211,

Thanks for brining this to our attention.

We are reaching out to the internal team to get more details on this behaviour. I will be update you once I hear back from the team.

0 Votes 0 ·