Any correlation between PipelineId and CorelationId in ADF

Question

Hello ,
Is there any relationship between pipelinerunid and correlationId in ADF v2?

We have enabled LogAnalytics in ADF v2 and now we need to backtrack a pipelinerun for which I have its PipelineRunid.

So how should I filter out the log data based on pipelinerunid?

There are some posts wherein they have mentioned that corelationid and pipelineid are same :
https://stackoverflow.com/questions/54034373/adf-log-analytics-how-to-correlate-multiple-pipelines
But that is not the case.

So wanted the help on the same

Answer 1

Hi @Nandan Hegde ,

PipelineID is the ID of the pipeline run.

CorrelationId is the unique ID for tracking a particular request.

Ref: Monitor and Alert Data Factory by using Azure Monitor

How should I filter out the log data based on PipelineRunId?

When we open the logs, we query the log tables ADFPipelineRun and ADFActivityRun by column PipelineRunId/RunID.

For example:

For more details, you could ref: Tutorial: Get started with Log Analytics queries.

Ref - https://stackoverflow.com/a/62526542/10653466

Hope this helps.

Answer 2

Can we have a query that shows the parent pipeline that failed and just that exact activity that failed within the parent pipeline. Is there a way to traverse from the activity that failed all the way back to the pipeline that triggered this activity.