question

balaa-uk avatar image
0 Votes"
balaa-uk asked ·

How do you process 'n' rows at a time?

I am ingesting data from SQL Server to an external system through Copy activity which accepts only 500 rows/records at a time. There has to be a wait period (let's say 5 minutes) before ingesting next set of data. How do we implement it in ADF? Are there any workarounds?

azure-data-factory
· 2
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @balaa-uk,

Just checking in to see if the below answer provided by @VaibhavChaudhari helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hello @balaa-uk,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.
Take care & stay safe!

0 Votes 0 ·

1 Answer

VaibhavChaudhari avatar image
0 Votes"
VaibhavChaudhari answered ·

If this is a Copy activity then at Source instead of selecting table directly, write a query that returns 500 records in loop with some delay in between, using loop.

To get the query logic, would suggest to post separate question with sql-server-general tag


Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav

·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.