Spark SQL DML statements with Delta tables

Serverless SQL 216 MVP

I am getting an unspecified error when running UPDATE, DELETE and MERGE sql statements on Delta tables when using Spark SQL APIs. I can use Scala APIs fine, I’m wondering if anyone else is having the same issue?

2 answers

Euan Garden 136 Reputation points

2020-05-28T01:39:01.077+00:00

You can see here that SQL based DML is mostly not supported in the LF version of Delta Lake, it will come in the future.

https://docs.delta.io/latest/quick-start.html#create-a-table

-Euan
Please sign in to rate this answer.

1 person found this answer helpful.
KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2020-05-28T22:59:19.153+00:00

Hi @AndyCutler-3530 ,

Following up to see if the above info from @Euan Garden was helpful. If this answers your query, please do consider to click "accept the answer" and/or "Up-Vote" as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2020-05-29T22:09:40.467+00:00

Hi @AndyCutler-3530 ,

Following up to see if the above info answers your query. If you have any further query do let us know.

Vaibhav Chaudhari 38,606 Reputation points

2020-05-30T19:57:12.32+00:00

Could some clarify - what is LF version here?

Euan Garden 136 Reputation points

2020-06-04T20:13:55.15+00:00

The Linux Foundation version of Delta Lake which is the OSS one as opposed to the one from Databricks
Sign in to comment
Andy Cutler 1 Reputation point

2020-05-30T15:09:53.677+00:00

Ah OK thanks. So the SQL DML statements only work on Databricks at the moment.
Please sign in to rate this answer.
Euan Garden 136 Reputation points

2020-06-04T20:14:47.443+00:00

A small subset do in LF but most do not.
Sign in to comment