question

AshwinMohan-2420 avatar image
0 Votes"
AshwinMohan-2420 asked AshwinMohan-2420 commented

Can adf pipeline level logs can be sent to data dog using streaming to event hub?

Hi there,
Is it possible to stream pipeline logs, for eg a copy activity fail log and send it to data dog by using the streaming to event hub feature ?
Is there any other way to do this?
I have understood data factory metrics can be send as-is to data dog.

azure-data-factory
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@AshwinMohan-2420 did my response answer your query? If it solved your issue please mark as accepted answer. If not, please let me know how I may better assist.

1 Vote 1 ·

Sorry for the delayed reply.
Thanks for the answer.

1 Vote 1 ·

1 Answer

MartinJaffer-MSFT avatar image
1 Vote"
MartinJaffer-MSFT answered AshwinMohan-2420 commented

Hello @AshwinMohan-2420 and welcome to Microsoft Q&A.

I don't know anything about "data dog", but I do know the details you seek are under "activityRuns", and I know that the logs can be streamed to event hub. In the portal, go to your Factory, then look under "diagnostic settings". You will also want "pipelineRuns" to correlate a given copy activity instance to a specific pipeline run.

122742-image.png



image.png (102.2 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks a ton for this.
I I used to think that activityruns as a metric, like the
count of activityruns.
Now I know, it provides the entire log insights.

2 Votes 2 ·