question

MariyaSusnerwala-3039 avatar image
0 Votes"
MariyaSusnerwala-3039 asked ShaikMaheer-MSFT commented

Copy latest files from SAP Server using Azure Data Facotry and File System Linked Service and store it in Azure Data Lake

I have a task to copy the newly added files everyday from SAP Server and store it in ADLS. There are 2 types files on the server (Recurring and Upfront) appended with date. I need to store the files in separate folder and everyday add the latest file from the SAP Server to the ADLS.

File name format: R_07292021.orc Recurring_08312021.orc U_07292021.orc Upfront_08312021.orc

Below are the steps I have taken so far

  1. Get Metadata Activity to get the list of files from the server


130856-image.png

  1. Use filter activity to separate the files based on the names, so filtering with the initial letter

130844-image.png

  1. I tried using the Foreach activity and If Condition, but it doesn't seem to be working.

I am not sure how to proceed from here. Any help is appreciated. Thank you!!




azure-data-factoryazure-files
image.png (55.9 KiB)
image.png (53.6 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MariyaSusnerwala-3039 ,

Following up to see is below answer helps you? If yes, please consider clicking on Accept Answer. Accepted answers helps us and community as well. Please let us know if any further queries. Thank you.

0 Votes 0 ·

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered ShaikMaheer-MSFT commented

Hi @MariyaSusnerwala-3039 ,

Thank you for posting query on Microsoft Q&A Platform.

Similar case to get latest file discussed in below links. Kindly check them.
https://docs.microsoft.com/en-us/answers/questions/136815/lates-file-extraction-azure-data-factory.html

You can refer below video also which explains same case step by step.
https://www.youtube.com/watch?v=QHeG36oqkPQ

Hope this is helpful. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks @ShaikMaheer-MSFT for the answer. The 2nd link for the solution helped me solve this issue.

0 Votes 0 ·

Hi @MariyaSusnerwala-3039 ,

Happy know that answer helped you. Thank you.

0 Votes 0 ·