what is the basic steps to improve azure databricks performance please help
Thanks for using Microsoft Q&A forum and posting your query.
While considering Azure Databricks performance, you will have to review below stages to know what are the bottlenecks that needs an attention.
Job and stage latency
Sum of task execution latency
Task metrics per stage
Streaming throughput and latency
Resource consumption per executor
For a detailed description of each of the above stages, please refer to this doc which has a clear explanation with images/samples: Troubleshoot performance bottlenecks in Azure Databricks
Hope this info helps.
Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
9 people are following this question.