Jobs are not progressing in the workspace

Problem

Jobs fail to run on any cluster in the workspace.

Cause

This can happen if you have changed the VNet of an existing workspace. Changing the VNet of an existing Azure Databricks workspace is not supported.

Review Deploy Azure Databricks in your VNet for more details.

Solution

  1. Open the cluster driver logs in the Azure Databricks UI.

  2. Search for the following WARN messages:

    19/11/19 16:50:29 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
    19/11/19 16:50:44 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
    19/11/19 16:50:59 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
    

    If this error is present, it is likely that the VNet of the Azure Databricks workspace was changed.

  3. Revert the change to restore the original VNet configuration that was used when the Azure Databricks workspace was created.

  4. Restart the running cluster.

  5. Resubmit your jobs.

  6. Verify the jobs are getting resources.