question

JohnAherne-0132 avatar image
0 Votes"
JohnAherne-0132 asked JoshuaAbad-1992 commented

Azure Monitor/Log Analytics - Get output of Data Factory Activities

I have several Data Factories tied to a log analytics workspace. At first glance, all the data appears to be there, but one thing that seems to be missing is the output from activities.

These outputs, particularly from copy activities are an important part of the metrics. Are they not included or am I missing something?

Thanks!

azure-data-factoryazure-monitor
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @JohnAherne-0132,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

Please suggest if you are using "Resource Specific" or "Azure Diagnostics" as the mode in Log Analytics settings. In Resource-Specific mode, diagnostic logs from Azure Data Factory flow into ADFPipelineRun, ADFTriggerRun, and ADFActivityRun tables. I can see there is output column in ADFActivityRun table, please suggest if this is not helping you in getting the desired results.

0 Votes 0 ·

I am using resource specific as the mode.

The output column only exists in ADFActivityRun and does not contain the output of the activity - It has some metadata, the inputs to the activity and any errors. There is nothing about the output of the activity itself (e.g. a copy activity does not show the rows copied and other data you get when you look at a copy activity through the ADF UI monitor)

0 Votes 0 ·

Any updates regarding with this one?

0 Votes 0 ·

1 Answer

HarithaMaddi-MSFT avatar image
0 Votes"
HarithaMaddi-MSFT answered JoshuaAbad-1992 commented

Thanks @JohnAherne-0132 for sharing the details. As of today, we cannot see the output of activity in the logs. I checked with Product team and they confirmed that the fix is in progress and it will be deployed soon.

Thanks for your patience!

· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for the update!

0 Votes 0 ·

@HarithaMaddi-MSFT Where would I go to check to see when this is implemented?

0 Votes 0 ·

@JohnAherne-0132 - Our team will share the update and details once we hear back more details on the fix from Product team. Thank you!

0 Votes 0 ·

@HarithaMaddi-MSFT It looks like this was implemented. I am now seeing the output of activities in log analytics.

Thanks for your help!

1 Vote 1 ·

Hi,

Was this ever fixed? The output column for all my activity runs is empty.

Thx,

0 Votes 0 ·
Show more comments