question

sakuraime avatar image
0 Votes"
sakuraime asked PRADEEPCHEEKATLA-MSFT edited

Azure databricks spark UI logs

what's the spark UI log history for the job ran for a cluster?

127869-image.png


azure-databricks
image.png (175.4 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT edited

Hello @sakuraime,

Thanks for the question and using MS Q&A platform.

The Spark UI is the web interface of a running Spark application to monitor and inspect Spark job executions in a web browser.

Apache Spark provides a suite of Web UI/User (aka Application UI or webUI or Spark UI) Interfaces (Jobs, Stages, Tasks, Storage, Environment, Executors, and SQL) to monitor the status of your Spark/PySpark application, resource consumption of Spark cluster, and Spark configurations.

For more details, refer to the below articles:

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

my questions is how long the logs are kept per cluster . Thanks

0 Votes 0 ·

Hello @sakuraime,

Events are stored for 60 days, which is comparable to other data retention times in Azure Databricks.

0 Votes 0 ·