question

MikeMcLeanShell-8864 avatar image
0 Votes"
MikeMcLeanShell-8864 asked PatrickYee-5128 commented

BUG: ADF 'Quick re-use' feature causes 'The databricks job of Dataflow completed, but the runtime state is either null or still InProgress.' Error

I saw the recent release of the Dataflow "Quick re-use" feature and immediately enabled it (the start-up time of pre-warmed clusters had been something that had been an issue for us for age)

Unfortunately, I think this feature is buggy.
Ever since enabling this setting, I've been intermittently having activity spontaneously hang and then error with:
"The databricks job of Dataflow completed, but the runtime state is either null or still InProgress."

It's happened on a variety of different dataflows, with nothing in common other than "they all use the same IR, which now has 'Quick re-use' enabled".

In case it's of use, one such run would be:

Pipeline Run Id: 0198122b-6bb1-4250-9fcc-3d28417366e7
Activity Run Id: 1141af06-376a-4554-97da-c36a79278113
"dataFlowETag": "0e006849-0000-0c00-0000-6065e9da0000"

But I've had numerous other instances of it happening.

azure-data-factory
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

More examples:

  • Pipeline RunId: 0198122b-6bb1-4250-9fcc-3d28417366e7
    • Activity RunId: 1141af06-376a-4554-97da-c36a79278113; dataFlowETag: 0e006849-0000-0c00-0000-6065e9da0000 (Above)

    • Activity RunId: fed793a2-595e-4578-a230-fe118684015e; dataFlowETag: 0e006249-0000-0c00-0000-6065e9da0000

  • Pipeline RunId: 1d69b761-6cf2-4a70-96a9-555184f2db09

    • Activity RunId: 17f058e0-51bb-4739-ad65-2ba751dda134; dataFlowETag: 3c000239-0000-0c00-0000-608819be0000

    • Activity RunId: b497dd6e-b55f-47e3-a131-813ecdb9d9d9; dataFlowETag: 3c001b39-0000-0c00-0000-608819be0000

0 Votes 0 ·

Hi @mikemcleanshell-8864,

Thanks for using Microsoft Q&A !!

I am checking internally with the products team on the same and will update you back on the same. Thanks for sharing your run-id's as well and this would be helpful. I will let you know in case I will need anything else from you.

Thanks
Saurabh

0 Votes 0 ·

Hi @mikemcleanshell-8864,

We haven't heard back from you. Just wanted to check if you are you still facing the issue? Please send us an email if you need help on creating a support ticket.

Thanks
Saurabh

0 Votes 0 ·
Show more comments
SaurabhSharma-msft avatar image
0 Votes"
SaurabhSharma-msft answered SaurabhSharma-msft commented

Hi @mikemcleanshell-8864,

The fix has been deployed in production for North Europe region. Please check and let me know if you see any issues.

Thanks
Saurabh

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Excellent news. I'll watch for any further re-ocurrences.
Alas it wasn't a particularly reliable error, so I can't explicitly test it, but I will monitor for a week and update you with the results.

0 Votes 0 ·

We've monitored for a week, and have seen no further issues. It does seem to have been resolved. Many thanks!

0 Votes 0 ·

@mikemcleanshell-8864 Great. Thanks for the confirmation.

0 Votes 0 ·
DavidHERNANDOMicropole-9463 avatar image
1 Vote"
DavidHERNANDOMicropole-9463 answered DavidHERNANDOMicropole-9463 commented

Hi,

We are facing the same issue on our subscription with adf resources in North Europe today the 8th of July 2021.

Are you deploying a new version or patch ?

Please Help!

Thanks!

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello,
I have the very same issue, I'm waiting for a situation update

0 Votes 0 ·

We could execute our pipelines by changing the location of the runtime to west europe...temporarily

0 Votes 0 ·