question

bhargavkandala-1177 avatar image
0 Votes"
bhargavkandala-1177 asked ShaikMaheer-MSFT commented

Load files fromADL Gen2 to SQLMI

Hello Everyone,
I have 30 files (.txt) in Azure Storage Explorer and i want to load these 30 files into 30 different tables in a SQL Managed Instance. Can someone please share if there is a link or document?
Thanks

azure-data-factory
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Any advice please or should i end up creating 60 datasets?
Thanks

0 Votes 0 ·

Hi @bhargavkandala-1177 ,

Following up to see is below answer helps you? If yes, Please consider click on Accept Answer. Accepted answers helps community as well.

0 Votes 0 ·

Hi @bhargavkandala-1177 ,

Just checking to see is below answer helps you? If yes, Please consider click on Accept Answer. Accepted answers helps community as well.

0 Votes 0 ·

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered ShaikMaheer-MSFT commented

Hi @bhargavkandala-1177 ,

Welcome to Microsoft Q&A Platform. Thank you for posting query here.

You can have your filename and table name mapping defined in some config table and then lookup that table using Lookup activity. use ForEach activity to iterate over that lookup data and inside ForEach activity use Copy activity

Your datasets inside Copy activity(source dataset & sink dataset) should be parameterized to dynamically points your file and table.

Since your are parameterizing your datasets you will use single dataset to point all files dynamically and single dataset to point to all tables dynamically.

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @bhargavkandala-1177 ,

Thank you for response. You can refer below documentation link which explains copy data from file to table.
https://docs.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-portal

Please Note, above documentation explains by considering one file. Same logic you need to implement in your case for multiple files using parameterized datasets to point to your files and tables dynamically.

You can refer below links to know how to parameterize datasets.
https://www.youtube.com/watch?v=9XSJih4k-l8

Hope this will help. Please let us know if any further queries. Thank you.

Please consider clicking on 130616-image.png button if this information is helpful.

0 Votes 0 ·