question

RamyaHarinarthini-7767 avatar image
RamyaHarinarthini-7767 asked ·

How to integrate/add more metrics & info into Ganglia UI in Databricks Jobs

As per https://docs.microsoft.com/en-us/azure/databricks/clusters/clusters-manage#monitor-performance, Ganglia metrics Collection Period Snapshot modifications can be done using init scripts.

Could you please help with pointers to modify by default Ganglia dashboard provided in Databricks jobs.

[Note: As we migrate from MSDN, this question has been posted by an Azure Cloud Engineer as a frequently asked question]

Sourced from MSDN – Azure Databricks


azure-databricks
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
PRADEEPCHEEKATLA-MSFT answered ·

Welcome to the Microsoft Q&A platform.

Happy to answer your questions

The pre-installed Ganglia UI is not modifiable in that way. If you want to create custom dashboards, you should set up your own monitoring system by using Azure Log Analytics.

One option is to use Azure Monitor + Power BI dashboards: https://docs.microsoft.com/en-us/azure/architecture/databricks-monitoring/

OR

Monitoring and Logging in Azure Databricks With Azure Log Analytics and Grafana: https://cloudarchitected.com/2019/04/monitoring-azure-databricks/

Hope this helps.

Sourced from MSDN – Azure Databricks


Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.