question

Adrian-5124 avatar image
0 Votes"
Adrian-5124 asked Adrian-5124 edited

DataFactory Alter Row into Sink

Hello!

I am deleting some blank rows from my data, I am using the 'Alter Row' function to after which I am running a isNull(COLUMNNAME) to get rid of the data.

It is appearing fine in the Alter Row box during the data preview, however after I try to sink the data into a CSV - I am still getting all the columns that are being deleted beforehand? Not sure what I am doing wrong here, would appreciate any tips.

Here are the screenshots:
127899-image.png


This is the data preview of my Alter Row transformation. After is the data preview of my Sink transformation, as I can see all the rows that are supposed to be deleted, or have the red 'X' next to them, are still featured during the sink.



127923-image.png


azure-data-factoryazure-synapse-analyticsazure-databricksazure-database-migration
image.png (141.9 KiB)
image.png (100.2 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

HimanshuSinha-MSFT avatar image
0 Votes"
HimanshuSinha-MSFT answered Adrian-5124 edited

Hello @Adrian-5124 ,
Thanks for the ask and using the Microsoft Q&A platform .
I checked your logic and it looks fine to me , it should work . Did you tried to trigger the pipeline and test .

Read more here . https://docs.microsoft.com/en-us/azure/data-factory/data-flow-alter-row

"Alter Row transformations will only operate on database, REST, or CosmosDB sinks in your data flow. The actions that you assign to rows (insert, update, delete, upsert) won't occur during debug sessions. Run an Execute Data Flow activity in a pipeline to enact the alter row policies on your database tables."


Please do let me know how it goes .
Thanks
Himanshu
Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members






· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

So I am unable to delete rows and sink to Blob storage with data factory? Whats the point of that if I can do all the other transformations and sink to blob haha

0 Votes 0 ·