question

JainAyushri-0042 avatar image
0 Votes"
JainAyushri-0042 asked JainAyushri-0042 commented

Databricks Cluster Logs Driver stdout - delayed in dbfs location

I have a databricks cluster with logging enabled in a dbfs location. My process requires to read the cluster logs, specifically the driver/stdout logs. This stdout is nothing but the console output which is also visible in UI : Clusters -> ClusterName -> Driver Logs -> Standard Output.
Every time, the UI shows me the stdout information very quickly but when I try to read the same from dbfs location, it's not yet present there and usually takes 4-5 minutes to reach there.
Is this delay expected? Is there any way to immediately sync the UI stdout and the dbfs stdout logs?

Enterprise Databricks configuration - 7.5 ML (includes Apache Spark 3.0.1, Scala 2.12)


azure-databricks
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @JainAyushri-0042,

Thanks for the question and using Microsoft Q&A platform.

As per the repro, even I noticed same behaviour.

We are reaching out to product team to get confirmation on this ask and will update you as soon as we have a response.

Thank you for your patience.

0 Votes 0 ·

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered JainAyushri-0042 commented

Hello @JainAyushri-0042,

According to the document - Logs are delivered every five minutes to your chosen destination.

It’s an expected behavior. I don't believe there is a way to circumvent the process.

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @PRADEEPCHEEKATLA-MSFT ,

Is there a way to read the data using databricks notebook from UI : Clusters -> ClusterName -> Driver Logs -> Standard Output ?

0 Votes 0 ·

Hello @JainAyushri-0042,

You can use magic cells - %sh cat to read the Driver Logs using databricks notebook.

100425-image.png

Hope this helps. Do let us know if you any further queries.

0 Votes 0 ·
image.png (195.2 KiB)

It's the same thing! Reading from dbfs - which has 5 minutes delay as per documentation.
Thanks though.

0 Votes 0 ·