For awhile now I have been trying to get this to work but no success...
My source is a view on prem sql server, say vw_Department
The data flow has source, alter row with upsert if set to true() and the sink Delta parquet has Allow upsert set to true
The keyColumn is also set for the sink, i.e. DepartmentID (However, this is dynamically retrieved, split($dfpKeyColumn, ',')
Anyway, the upsert if seems to be working fine because if I edit the source table, i.e. update a row or insert a row iinto the table, then the Sink parquet Delta reflects those changes...
Then I try to allow for if a row is deleted at source, i.e. a row is removed from the table in source on-prem sql server.
So I add the Delete If and Allow Delete accordingly...
When a row is removed from the source, then in Sink, still I see all the rows including the deleted ones. Whereas I should only see the upsert or unchanged ones.
I know I have to place a condition for the Deleted If but not sure what to put for it to make this delete work fine.
Noe that I can not put a condition like "Name" == "test" because I want to remove the row from Sink if it is deleted at source.
I placed the upsert If after the Delete If and even changed their priority other way round but still not solved the issue yet