Share via

databricks integration with synapse

Vineet S 185 Reputation points
2024-05-13T16:06:41.3633333+00:00

hey,

is there any third party tool is available which will integrate synapse with databricks

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,477 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Smaran Thoomu 10,875 Reputation points Microsoft Vendor
    2024-05-13T18:18:01.1066667+00:00

    Hi @Vineet S

    Thanks for the question and using MS Q&A platform.

    One way to integrate Azure Synapse with Databricks is by using PolyBase data virtualization. PolyBase allows you to create a logical data warehouse where data in Azure Synapse is combined with data from other Azure and on-premises analytical data stores like HDInsight, Azure Cosmos DB, or streaming data flowing into Data Lake Storage from Stream Analytics or Event Hubs. This approach simplifies business user access to data and adds new value to what you already know in your data warehouse.
    Here is the document that might be helpful on how to use PolyBase to integrate Azure Synapse with Databricks https://learn.microsoft.com/en-us/azure/databricks/archive/azure/synapse-polybase

    Another way to integrate Azure Synapse with Databricks is by using the Azure Synapse Spark connector. This connector enables you to connect to Databricks and transfer large data sets between the Apache Spark runtime and the Dedicated SQL pool. To move data between Databricks and Synapse, you can also use the Azure Synapse Dedicated SQL Pool Connector for Apache Spark.
    Reference: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/synapse-spark-sql-pool-import-export?tabs=scala%2Cscala1%2Cscala2%2Cscala3%2Cscala4%2Cscala5

    Also, you can find more information on Databricks integration with Azure Synapse in this video link please refer.

    Hope this helps. Do let us know if you any further queries.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.