Dataflow Activity giving output in different formats - "runStatus" vs. "SubStatus"

Rajas Thakur 56

Hello,

We have some excel files, which are loaded into the Datalake as .txt files through CopyData Activity.
After that, We have a Data Flow activity to copy data from my DataLake to Azure SQL DB. We have implemented some logic after that for logging, for which we need to access the output of the Data Flow Activity. Ideally, we should get "runStatus" from the JSON output of the activity. But for some reason, we are getting "SubStatus" instead of "runStatus" in the output in some instances.

Does anyone know what is the reason for getting "SubStatus" instead of "runStatus"?
Is there any way in which we can make sure we always get "runStatus" from the DataFlow activity output?

Thanks

Rajas Thakur 56 Reputation points

2022-05-02T16:19:58.15+00:00

Hello @AnnuKumari-MSFT ,

Thank you for your reply.
Yes, your understanding is correct. We are running this DataFlow block inside a foreach block for multiple tables. We are expecting this output from the DataFlow Activity in each run:

For some instances, we are getting output like this:

Because we are getting "SubStatus" instead of "runStatus" in some cases, the blocks downstream fail causing the whole pipeline to fail. However, when we run the pipeline again for the specific file that caused this error, it goes through fine.

I checked the DataFlow activity in both instances (when getting JSON output as "SubStatus" and "runStatus") for input parameters it gets and they were the same. Also the DataFlow activity seems to be completed just fine (the data gets inserted in the tables),but the JSON output of that activity is different.

Thank you for your help!
Rajas Thakur 56 Reputation points

2022-07-07T20:40:35.51+00:00

Hello @AnnuKumari-MSFT ,

Hope you are well.

We have recently started to see that this issue is occurring again. The data flow activity is not giving outputs in the expected formats ('runStatus').

Can you kindly check if this is a known issue with the internal product team and if there is any possible hotfix for this issue?

Thanks,
Rajas Thakur
AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-07-08T10:22:26.407+00:00

Hi @Rajas Thakur ,
I have reached out to the owner team of the incident for this issue . I will let you know if I hear back anything from them . Thankyou for your patience!
AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-07-13T03:57:16.797+00:00

Hi @Rajas Thakur ,
Does the issue still persist for you?
Rajas Thakur 56 Reputation points

2022-07-28T15:27:08.473+00:00

Hi @AnnuKumari-MSFT ,

The last we got this issue was around 2-3 weeks ago.
We haven't received it after that.

Thanks. Appreciate your help.

Regards,
Rajas

Accepted answer

AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-05-02T10:17:24.18+00:00

Hi @Rajas Thakur ,

Welcome to Microsoft Q&A platform and thankyou for posting your query.

Going through your query, it looks like the JSON output of dataflow is giving you SubStatus as the property instead of runStatus Property. Please let me know if my understanding is incorrect.

Could you please share the screenshot of the same so that I can approach the internal product team to investigate further. I tried to reproduce your scenario but didn't face similar challenge. For me , dataflow output gave runStatus property. Thanks in advance !
Please sign in to rate this answer.

1 person found this answer helpful.
AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-05-04T07:52:09.687+00:00

Hi @Rajas Thakur ,

Thankyou for providing detailed screenshot . I escalated this issue to the internal Product team and got to know they have recently deployed a hotfix for this issue. Could you please confirm if you are still experiencing the issue or has it been resolved at your end? Thanks in advance!

If the issue is resolved, please click Accept Answer as accepted answer helps community as well.

Rajas Thakur 56 Reputation points

2022-05-04T19:46:11.903+00:00

Hello @AnnuKumari-MSFT ,

Thanks for the update. The last time we saw this issue was on April 29th. After that, we haven't encountered this issue so I believe that the hotfix that you have mentioned should have resolved the issue.

Thank you for your timely updates. I appreciate it.

AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-05-05T04:25:44.577+00:00

Hi @Rajas Thakur ,
Thanks for the confirmation !
Sign in to comment

Dataflow Activity giving output in different formats - "runStatus" vs. "SubStatus"

0 additional answers