We are looking into a possibility to move some notebooks from Databricks to Synapse Studio. However, it seems that it is impossible in Synapse Spark notebooks to query tables with uniqueidentifiers?
Example table:
CREATE TABLE dbo.test
(id uniqueidentifier null
, ts datetimeoffset not null
,data float null
)
Create a spark dataframe in notebook:
val df = spark.read.synapsesql("testdb.dbo.test")
display(df)
The display() command returns error:
Error : com.microsoft.spark.sqlanalytics.exception.SQLAnalyticsConnectorException: com.microsoft.sqlserver.jdbc.SQLServerException: Columns with UniqueIdentifier types are not supported in external tables.
I thought about creating a view where the uniqueidentifier would be a varchar, but views are not supported by synapsesql().
Do I have any other way to get this table into Spark notebook for some processing, or should we just forget about Synapse studio for now?