question

PriyaJha-3992 avatar image
0 Votes"
PriyaJha-3992 asked NandanHegde-7720 commented

How to identify status of an activity in Azure data factory

Hi All,

I was following below architecture for removing duplicate activities to get the error message of every activity for logging purpose.
113932-108736-image.png

For logging error at every activity level i used the following code to concat error messages from every activity
@concat(activity('Act_1').error?.message,'^|',activity('Act_2').error?.message,'^|',activity('Act_3').error?.message,'^|',activity('Act_4').error?.message)

But because we are using Skipped along with completion dependency, the upper concat expression fails with the following error when an activity is skipped:
The expression 'concat(activity('Act_2').error?.message)' cannot be evaluated because property 'error' cannot be selected.

Is there any way, i can get error messages for every activity in the above architecture?

Also is there any flag or way to identify whether an activity went through a skipped ,or completion case if all 2 flows are mapping to same output?


azure-data-factory
108736-image.png (163.6 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I almost forgot your other ask @PriyaJha-3992 .

When an activity is skipped, we can't exactly fetch its status. However, the absence of status is also a strong indicator.

This means I can combine the null-safe operator, and the coalesce function, to figure out the status of an activity.

 @{coalesce(activity('Lookup1')?.Status,'Skipped')}

Is this close enough to your ask of determining whether skipped or completed?


1 Vote 1 ·

1 Answer

NandanHegde-7720 avatar image
1 Vote"
NandanHegde-7720 answered NandanHegde-7720 commented

Hey,
Since you have used Skipped logic, there would not be any output because of which you are getting the error.
You can validate whether there is any output or not by using '?':
activity('Act_2')?.error?.message

My guess is there is no flag available based on my knowledge.

@MartinJaffer-MSFT @KranthiPakala-MSFT @HimanshuSinha-MSFT @nasreen-akter : please correct me if I am wrong or any other thoughts on the same.


· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

So close, @PriyaJha-3992 , you almost had it. @NandanHegde-7720 is correct.

When the activity has not run, it cannot make an error. The ?. makes a thing null-safe.

You wrote

 activity('Act_2').error?.message

Nandan wrote

 activity('Act_2')?.error?.message

The difference is where the ?. is located. With the ?. before message it is anticipating whether message is null. However it cannot get to that point if error does not exist / is null. This is why placing the ?. before error fixes the issue. The ?. before message may not be necessary.

 activity('Act_2')?.error.message

Thank you for the quick answer Nandan. :-)

1 Vote 1 ·

Hey @MartinJaffer-MSFT ,
Based on the above Arch, my guess is there should be .? before message as well because lets consider the scenario wherein the first 2 activities are successful and the last one fails .
And since there is no error message for the initial 2 ones the concat statement might fail without .?

0 Votes 0 ·