question

MaheshMadhusanka-2060 avatar image
0 Votes"
MaheshMadhusanka-2060 asked ·

How Can i Increment Data Append to initial File in Data Factory

Hi Team,

Currently i have copied Single bulk table from SQL database to Datalake 2, But now i'm stuck on the how to increment daily data(insert and Modified) on that bulk file, How can i do? What is the easiest way to do that? Your immediate response much appreciate.


azure-data-factory
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered ·

Hi @MaheshMadhusanka-2060 ,

Welcome to Microsoft Q&A and thank you for your query.

Unfortunately appending to an existing blob using ADF copy activity is not supported. I would recommend you to please up-vote and/or comment on the below existing similar feature request from other users. Up-voting/commenting will help to increase the priority of the feature request.

Existing feature request: https://feedback.azure.com/forums/270578-data-factory/suggestions/36163726-append-replace-upsert-the-outgoing-data-from-copy

Additional info:

Please have a look at this SO thread and see if that helps as a workaround for your scenario: Is there any workaround for using Append blob in Azure Data Factory copy?


Hope this info helps.



Thank you

Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.




· 5
10 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @KranthiPakala-MSFT ,


Thank you for your response and info, You mean we don't have any option to new incremental data append on the exsisting bulk table csv file on azure data lake? is it?


0 Votes 0 ·

Hi MaheshMadhusanka-2060,

Thanks for your response. Yes, append data to an existing file is not support by ADF copy at this time. Please feel free to up-vote/comment on the above feature request thread to increase the priority of the feature request.

0 Votes 0 ·

Hi @KranthiPakala-MSFT

Thank you for your response, Yes I already Vote it, However Can i use Union function on data flow or is there have a any merge feature on that, for this concern i have already raised new Q & A Can you please advise on this as well?

https://docs.microsoft.com/en-us/answers/questions/32743/how-to-setup-copy-activity-with-merge-data-flow-in.html

0 Votes 0 ·
Show more comments