Activity stuck in queue status

Gużewski, Jacek 75

Starting from today morning, I am facing problem that any pipeline run, by trigger or manually or in debug mode, doesn't start. To be precise, pipeline starts ("In progress", but first activity stays in "Queue" status). Doesn't matter what type the first activity is - script to database , GetMetadata from file from storage account, etc.).

I use own integration runtime with sub-type MV Net.

It is not a problem with performance because I start only one pipeline at the same time.

Problem started between 3:00 to 5:20 my local time.

Trigger run at 2:00 went ok, but net one at 5:20 had this problem.

PRADEEPCHEEKATLA-MSFT 78,986 Reputation points Microsoft Employee

2024-04-30T12:51:23.5033333+00:00
@Gużewski, Jacek - Thanks for the question and using MS Q&A platform.

It seems like your pipeline is stuck in the "Queue" status and the first activity is not starting. There could be various reasons for this issue, such as hitting concurrency limits, service outages, network failures, and so on.

To resolve this issue, you can try the following steps:

If your pipeline has a concurrency policy, verify that there are no old pipeline runs in progress.

Go to the ADF authoring canvas, select your pipeline, and determine if it has a concurrency property assigned to it. If it does, go to the Monitoring view, and make sure there's nothing in the past 45 days that's in progress. If there is something in progress, you can cancel it, and the new pipeline run should start.

It is possible that your run was impacted by a transient network issue, credential failures, services outages, etc. If this happens, Azure Data Factory has an internal recovery process that monitors all the runs and starts them when it notices something went wrong. You can rerun pipelines and activities as described here: Rerun pipelines and activities. This process happens every one hour, so if your run is stuck for more than an hour, create a support case.

For more details, refer to Azure Data Factory: Pipeline status is queued or stuck for a long time.

If none of the above steps work, please provide more information about your pipeline and the activities involved and ADF region so that we can investigate further.

Hope this helps. Do let us know if you any further queries.
PRADEEPCHEEKATLA-MSFT 78,986 Reputation points Microsoft Employee

2024-05-02T03:27:11.7566667+00:00

@Gużewski, Jacek - We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Gużewski, Jacek 75 Reputation points

2024-05-06T05:44:31.3766667+00:00

Hi,

I resolve the problem.

I found in Service Health issue in DataFactory with the status - resolved:
Important was a part: "with Copy Compute Scale enabled".

I updated the settings for IR switching this off and on again and all started to work correctly,
PRADEEPCHEEKATLA-MSFT 78,986 Reputation points Microsoft Employee

2024-05-06T06:05:52.1466667+00:00

@Gużewski, Jacek - I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Accepted answer

PRADEEPCHEEKATLA-MSFT 78,986 Reputation points Microsoft Employee

2024-05-06T06:05:36.55+00:00

@Gużewski, Jacek - I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask: Activity stuck in queue status

Solution: I resolve the problem.

I found in Service Health issue in DataFactory with the status - resolved: Important was a part: "with Copy Compute Scale enabled".

I updated the settings for IR switching this off and on again and all started to work correctly,

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.
Gużewski, Jacek 75 Reputation points

2024-05-09T06:06:46.9133333+00:00

...........
Sign in to comment

Share via

Activity stuck in queue status

0 additional answers