question

SarahRobert-3529 avatar image
0 Votes"
SarahRobert-3529 asked SarahRobert-3529 commented

Move file in Azure data Factory

Hi
I was following this post https://docs.microsoft.com/en-us/answers/questions/57483/move-file-rather-than-copy-in-azure-data-factory.html
and asked a question there too.


I need to move files in ADF and used this create a new file and delete the old file approach. I don't need to read the file so thought copying like this be fine.

The only problem with this approach I am seeing is that I am losing the timestamp of the file because of creating a new file and deleting it.

Is there no way in ADF that we can move the original file without reading it and thus able to maintain the timestamp of the file? This is the last step in my pipeline and I really don't want to take this step to logic app. Unless there is a way for me to call the logic app within ADF?

Thanks

azure-data-factory
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@SarahRobert-3529 if my response answer your query, please mark as accepted answer. If not let me know how I may better assist.

0 Votes 0 ·

If you found your own solution, please share here with the community.

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered SarahRobert-3529 commented

Hello @SarahRobert-3529 and welcome to Microsoft Q&A.

There are a couple things you can try.

First, use binary type dataset, instead of a more specific one like CSV, JSON, etc. The binary does not attempt to parse what is inside the file.

Also, you can try using settings to preserve some of the metadata.

98012-image.png

The post you linked is a little stale. There is now a way to make the delete part of the copy action. You need not make the delete a separate activity.

97959-image.png


image.png (96.7 KiB)
image.png (139.2 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Apologize for the late answer, Wanted to update you. I still have not had a chance to try your suggestion. I shall get try and see if that works for my solution in a few days and get back to you. Thank you soo much.

0 Votes 0 ·

@SarahRobert-3529 did it work?

0 Votes 0 ·

HI @MartinJaffer-MSFT appreciate your patience. I circuled back to this and here is what I observed.

1) I created my datasets with binary type and I am not seeing the option to Preserve attributes in the settings tab like the way you are showing.
110377-image.png

2) We are still talking about deleting the files and creating new one at a different location. My original question was geared towards, a move of files. Is there a way to move files without deleting them from the original location? So we can preserve the original timestamps?

Again appreciate your patience and help with this.

0 Votes 0 ·
image.png (43.1 KiB)