How to Calculate the Number of Cores in a Cluster
If your organization has installed a metrics service on your cluster nodes, you can view the number of cores in an Azure Databricks cluster in the Workspace UI using the Metrics tab on the cluster details page.
If the driver and executors are of the same node type, you can also determine the number of cores available in a cluster programmatically, using Scala utility code:
sc.statusTracker.getExecutorInfos.lengthto get the total number of nodes. The result includes the driver node, so subtract 1.
java.lang.Runtime.getRuntime.availableProcessorsto get the number of cores per node.
Multiply both results (subtracting 1 from the total number of nodes) to get the total number of cores available:
java.lang.Runtime.getRuntime.availableProcessors * (sc.statusTracker.getExecutorInfos.length -1)