question

RubnSouto-1550 avatar image
0 Votes"
RubnSouto-1550 asked HarithaMaddi-MSFT answered

Too long "Queue" time using SAP ECC connector within Data Factory

48346-image.png



Greetings:

It happens occasionally, "copy data" activity gets stucked in "Queue" state for a long time. I send an example.

What can I do to troubleshoot it? Is there a solution?

Thank you.

Regards.

azure-data-factory
image.png (51.6 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you very much, i'll try the solutions and let you know.

1 Vote 1 ·

Hi @RubnSouto-1550,

Welcome to Microsoft Q&A Platform. Thanks for posting the query. I looked at similar issues from customers and observed that one of the below reasons can be a root cause for the issue.

  • Upgrading IR to latest version

  • Increasing the "Limit Concurrent jobs" in IR configuration

If above changes did not help in resolving the issue, please share report id along with pipeline run id with us. In the integration runtime configuration manager, please click on diagnostics icon, reproduce the error and then click on ‘Send logs’. Please share report id with us to enable viewing logs.

48690-image.png

Ref: issue

Looking forward for your response!

0 Votes 0 ·
image.png (139.6 KiB)

Hi @RubnSouto-1550,

We have not received a response from you. Are you still facing the issue? If you found a solution, would you please share it here with the community? Otherwise, let us know above details and we will continue to engage with you on the issue.

0 Votes 0 ·

Hi @HarithaMaddi-MSFT

Increasing the value of "Limit Concurrent Jobs" from 6 to 10 initially reduced a lot the number of cases, but last 2 days it's getting worse. Should I increase it more? I'm not sure about the performance issues this action could trigger.

FYI we are running around 180 "copy data" activities with SAP ECC connector every night. We use a "foreach" activity, allowing up to 3 parallel runs.

50428-image.png

So I believe you are right pointing to a concurrency issue, but... apparently the settings are right.

Now I'm gonna update IR, activate Logging function and let you know.

Thank you very much.


0 Votes 0 ·
image.png (5.9 KiB)

1 Answer

HarithaMaddi-MSFT avatar image
0 Votes"
HarithaMaddi-MSFT answered

Thanks @RubnSouto-1550 for sharing the update. As per the document, when the processor and available RAM aren't well utilized, but the execution of concurrent jobs reaches a node's limits, scale up by increasing the number of concurrent jobs that a node can run. If the node capacity also exceeds i.e., when issue persists after increasing the limit, then scale-out approach as suggested i.e., to increase nodes is preferred.

Looking forward for your update on the issue!


· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @RubnSouto-1550,

We have not received a response from you. Please suggest if above shared details are helpful. Otherwise, let us know and we will continue to engage with you on the query.

0 Votes 0 ·

Hi @RubnSouto-1550,

We still have not heard back from you. Following up to check if above shared details are helpful. Otherwise, let us know and we will continue to engage with you on the query.

0 Votes 0 ·