question

30289318 avatar image
0 Votes"
30289318 asked PRADEEPCHEEKATLA-MSFT commented

Update Synapse Analytics Table from Databricks

Is it possible to update a table of Synapse Analytics (Data Warehouse) from Azure Databricks?
Apparently using Pyspark, Scala or SparkR there is no update option. Besides, using Spark SQL the Update syntax does not work.

azure-synapse-analyticsazure-databricks
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
2 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT commented

Hello @30289318,

Welcome to the Microsoft Q&A platform.

You can update the Azure Synapse Analytics tables from Azure Databricks using overwrite mode.

Overwrite mode means that when saving a DataFrame to a data source, if data/table already exists, existing data is expected to be overwritten by the contents of the DataFrame.

As per the repro, I'm able to update the Azure Synapse Analytics Table from Azure Databricks.

Reading the table data from Azure Synapse Analytics using Azure Databricks:

100982-image.png

Inserting the data into table of Azure Synapse Analytics

100991-image.png

Updating the columns in the table of Azure Synapse Analytics

101001-image.png

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.


image.png (60.6 KiB)
image.png (83.7 KiB)
image.png (84.2 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @30289318,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hello @30289318,

Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.
Take care & stay safe!


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

1 Vote 1 ·

tried the provided solution but it didn't helped. Throwed an error saying, "
com.databricks.spark.sqldw.SqlDWSideException: Azure Synapse Analytics failed to execute the JDBC query produced by the connector."
could you please help, I am trying to Update a synapse table from the pyspark dataframe that I have in Databricks.
I am having a synapse table with columns A,B,C,D and I have a Dataframe that has the values for column D that I need to update in the Synapse Table.

0 Votes 0 ·

Hello @NaveenKumar-0720,

Thanks for the question and using MS Q&A platform.

I see you have already created a new thread: https://docs.microsoft.com/en-us/answers/questions/781726/accessing-synapse-table-from-databricks-using-pysp.html let us continue the discuss on the new thread.

0 Votes 0 ·