question

LilyLin-8573 avatar image
0 Votes"
LilyLin-8573 asked LilyLin-8573 commented

Azure Data Factory - Copy Activity from SHIR Stuck in "Queued"

The data sources connected to 1 of the SHIR is stuck in Queued during a copy task.
I tried creating just a simple copy of 1 table from an On premise ODBC source to Azure Blob (compressed parquet). It's stuck in Queued.

This is happening for both Scheduled and Manual Triggers as well as Debug.

Test Pipeline I created:
Run ID: 168dec06-a254-405a-97d4-e0b6b720772b (Manual Trigger)
Location: Eastern US 2

This wasn't an issue until beginning of Nov, and it's occurring more consistently the past few days.

azure-data-factory
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @LilyLin-8573,

Welcome to the Microsoft Q&A Platform.

This issue looks strange. Can you please try enabling retry options in copy activity and if the issue persists. please share data factory name, region, and report id using the below steps so that our team can access SHIR logs for further investigation.

How to send logs to investigate?

In the integration runtime configuration manager, please click on the diagnostics icon, reproduce the error and then click on ‘Send logs’.

You will be able to see the report id, please share that with us to enable viewing logs.

40677-image.png

Looking forward for your response!

0 Votes 0 ·
image.png (97.6 KiB)
LilyLin-8573 avatar image LilyLin-8573 PRADEEPCHEEKATLA-MSFT ·

Hi @PRADEEPCHEEKATLA-MSFT ,

Unfortunately, I do not have access to the configuration manager and when I tried to Query the log it says it failed
40833-image.png


I notice that this Queue Length counter just keeps going up when a new run is triggered, but it doesn't go down when the pipeline is Cancelled or Failed.
I don't think the Timed Out and Queued Activities are being cleared from this Queue even though the entire Pipeline is Cancelled / Failed.

Is that an expected behavior? Is there a way clear the Queue Length (without restarting windows machine) ?


40806-image.png


0 Votes 0 ·
image.png (19.7 KiB)
image.png (30.6 KiB)

Hi @LilyLin-8573,

When you say "Unfortunately, I do not have access to the configuration manager", could you please request your team members who have permissions to send SHIR logs for further investigation.

0 Votes 0 ·
Show more comments

Hi. We are also experiencing a similar issue to this which started about 4+ days ago.

0 Votes 0 ·

0 Answers