question

Anshal-7634 avatar image
0 Votes"
Anshal-7634 asked KranthiPakala-MSFT commented

databricks performance

what is the basic steps to improve azure databricks performance please help

azure-databricks
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @anshal-7634,

Thanks for using Microsoft Q&A forum and posting your query.

While considering Azure Databricks performance, you will have to review below stages to know what are the bottlenecks that needs an attention.

  • Job and stage latency

  • Sum of task execution latency

  • Task metrics per stage

  • Streaming throughput and latency

  • Resource consumption per executor

For a detailed description of each of the above stages, please refer to this doc which has a clear explanation with images/samples: Troubleshoot performance bottlenecks in Azure Databricks


Hope this info helps.



  • Please don't forget to click on 130616-image.png and upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @anshal-7634,

Just checking in to see if the above response was helpful. If it answers your query, please do consider to click 130616-image.png and upvote 130671-image.png for the same as it could be beneficial to other community members. And, if you have any further query do let us know.

Thank you

0 Votes 0 ·