question

AndyDCutler avatar image
1 Vote"
AndyDCutler asked ·

Spark SQL DML statements with Delta tables

I am getting an unspecified error when running UPDATE, DELETE and MERGE sql statements on Delta tables when using Spark SQL APIs. I can use Scala APIs fine, I’m wondering if anyone else is having the same issue?

azure-synapse-analytics
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

euangMS avatar image
1 Vote"
euangMS answered ·

You can see here that SQL based DML is mostly not supported in the LF version of Delta Lake, it will come in the future.

https://docs.delta.io/latest/quick-start.html#create-a-table

-Euan

· 4 · Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @AndyCutler-3530 ,

Following up to see if the above info from @euangMS was helpful. If this answers your query, please do consider to click "accept the answer" and/or "Up-Vote" as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

0 Votes 0 · ·

Hi @AndyCutler-3530 ,

Following up to see if the above info answers your query. If you have any further query do let us know.

0 Votes 0 · ·

Could some clarify - what is LF version here?

0 Votes 0 · ·
euangMS avatar image euangMS VaibhavChaudhari ·

The Linux Foundation version of Delta Lake which is the OSS one as opposed to the one from Databricks

0 Votes 0 · ·
andycutler avatar image
0 Votes"
andycutler answered ·

Ah OK thanks. So the SQL DML statements only work on Databricks at the moment.

· 1 · Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

A small subset do in LF but most do not.

0 Votes 0 · ·