question

AnjaliMaithani-7687 avatar image
0 Votes"
AnjaliMaithani-7687 asked ShreyasiSrivastava-1468 commented

Iteration and pagination to call rest api in adf

Hello Team,

I have created a copy activity which has source for REST API and Sink as Azure SQL DB. It work for the 1000 records.

I need to apply pagination to have the full data from Rest API and also iterate the loop.

From which activity and properties i can achieve this.
Please help ASAP.

Regards,
Anjali Maithani

azure-data-factory
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@AnjaliMaithani-7687 did my response solve your issue? If so, please mark as answer, otherwise let me know how I may better assist you.

0 Votes 0 ·

Hi, did your question got resolved. Could you put the solution here as I have same issue. I am not sure how to put the copy activity inside a loop in tis case.

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered MartinJaffer-MSFT commented

Hello @AnjaliMaithani-7687 and welcome to Microsoft Q&A.

With REST as source, you can add pagination rules in the copy activity source tab. The way to paginate depends upon how the source behaves. Please see this doc for details:

https://docs.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support

If yours does not behave like that, you can try parameterizing the dataset and put the copy activity inside a loop.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

How can i define parameter ? can you please describe the steps.

Regards

0 Votes 0 ·

@AnjaliMaithani-7687
Datasets and Linked services can be parameterized. You only need to do one of them.

For the dataset, open the dataset and go to the parameters tab...
84068-image.png
Then go to the connection tab and add a reference to it in the relative url field...
84114-image.png

For the linked service you can make it look like this:
84122-image.png


0 Votes 0 ·
image.png (25.3 KiB)
image.png (69.5 KiB)
image.png (155.3 KiB)