Networking

This article introduces networking configurations for the deployment and management of Azure Databricks accounts and workspaces.

Azure Databricks architecture overview

Azure Databricks operates out of a control plane and a compute plane.

• The control plane includes the backend services that Azure Databricks manages in your Azure Databricks account. The web application is in the control plane.
• The compute plane is where your data is processed. There are two types of compute planes depending on the compute that you are using.
  • For classic Azure Databricks compute, the compute resources are in your Azure subscription in what is called the classic compute plane. This refers to the network in your Azure subscription and its resources.
  • For serverless compute, the serverless compute resources run in a serverless compute plane in your Azure Databricks account.

For additional architecture information, see Azure Databricks architecture overview.

Secure network connectivity

Azure Databricks provides a secure networking environment by default, but if your organization has additional needs, you can configure network connectivity features between the different networking connections shown in the diagram below.

1. Users and applications to Azure Databricks: You can configure features to control access and provide private connectivity between users and their Azure Databricks workspaces. See Users to Azure Databricks networking.
2. The control plane and the classic compute plane: Classic compute resources, such as clusters, are deployed in are in your Azure subscription and connect to the control plane. You can use classic network connectivity features to deploy classic compute plane resources in your own virtual networks and to enable private connectivity from the clusters to the control plane. See Classic compute plane networking.
3. The serverless compute plane and storage: You can configure private and dedicated connections from serverless compute to storage. See Serverless compute plane networking.

You can configure Azure storage networking features, such as private endpoints to secure the connection between the classic compute plane and Azure storage. For more information, see Grant your Azure Databricks workspace access to Azure Data Lake Storage Gen2 and Networking recommendations for Lakehouse Federation.

Connectivity between the control plane and the serverless compute plane is always over the Azure network backbone and not the public internet.