question

NagaSri-9719 avatar image
0 Votes"
NagaSri-9719 asked ShaikMaheer-MSFT commented

How to Import incremental load form On premises DB into Synapse workspace

Hi Team,

I need to Import incremental load form On premises DB into Synapse workspace. Please provide leads.


Regards,
NagaSri

azure-data-factoryazure-synapse-analyticsazure-data-lake-storage
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @NagaSri-9719 ,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hi @NagaSri-9719 ,

Following up to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·
ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered

Hi @NagaSri-9719 ,

Thank you for posting your query on Microsoft Q&A Platform.

You can leverage Synapse Pipelines or Azure data factory pipelines to incrementally load On-primes DB in to Synapse.

Please check below documentation links which helps you figure out logic of implementation for same.

Please Note:
- To connect with on-prem data you need to configure self-hosted IR. Click here to know more about it.
- You need consider on-prem data as your source and Synapse as your sink in all above documentations logic.

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KuldeepChitrakar-0966 avatar image
0 Votes"
KuldeepChitrakar-0966 answered KuldeepChitrakar-0966 edited

I would use below approach

  1. Self hosted IR on your onPremise Network

  2. Use ADF Metadata Driven Copy Activity to incrementally read data from on premise and load that into files

  3. Store those files in ADLS

  4. Use Polybase to read these files and leverage ELT to transform and load the data in Synapse Dedicated Pool or you can use Spark pool to further process these files.


Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.