Azure subscription Quota Exceeded

Question

I have asked and granted for Total Regional vCPUs of 16.

Now I am trying to create 1 cluster with 3 workers in Azure Databricks with the use of Standard_DS3_v2 (14 G memory and 4 cores) and I get quota the following error

Azure Quota Exceeded Exception: Error code: QuotaExceeded, error message: Operation could not be completed as it results in exceeding approved standardDADSv5Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: GermanyWestCentral, Current Limit: 0, Current Usage: 0, Additional Required: 8, (Minimum) New Limit Required: 8.

What am I doing wrong?

Thanks

Answer 1

Hi Elias,

Based on the error you received, it appears you selected Standard_D8ads_v5 for your Driver and you don't have any Quota for that VM size family in Germany West Central region. Since this requires 8 vCPUs and your current limit is 0, it was unable to create the Driver for your cluster.

If you want to use Standard_D8ads_v5 for your driver, what you can do is request quota increase for Standard DADSv5 Family vCPUs to at least 8 vCPUs for Germany West Central. Additionally, please make sure Total Regional vCPUs for Germany West Central is high enough for the total number of your workers + your driver + any other usage (VMs, etc.) you may have.

Link to Quotas in portal:

https://portal.azure.com/#view/Microsoft_Azure_Capacity/QuotaMenuBlade/~/myQuotas

Another option would be to use a different VM size for your Driver. You can use same size for your Driver as you are using for your Workers if you want. The same rules apply--you need to have enough available vCPUs for total of each individual VM size family AND enough Total Regional vCPUs.

Please click Accept Answer and upvote if the above was helpful. If something is unclear please add a comment and I'll assist.

Thanks.

-TP