How to Use Dynamic values in pagination Rule (Azure data factory)?

Question

I'm using pagination rule in copy data activity from rest endpoint to blob storage.

I have applied pagination rule with dynamic value like this:

AbsoluteUrl = @replace('$.nextLink','old_base_url','new_base_url')

But it is still accessing the old_base_url which is I'm getting in response, doesn't replace it with new through string function replace.

Error in data factory:

ErrorCode=UserErrorFailToReadFromRestResource,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred while sending the request.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.Http.HttpRequestException,Message=An error occurred while sending the request.,Source=mscorlib,''Type=System.Net.WebException,Message=Unable to connect to the remote server,Source=System,''Type=System.Net.Sockets.SocketException,Message=A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond xx.xx.xx.xx:443,Source=System,'", "failureType": "UserError", "target": "cdr", "details": [] }

@KranthiPakala-MSFT
@SubhashVasarapu-8577

I have also asked this here
https://github.com/MicrosoftDocs/azure-docs/issues/56266

Answer 1

Hi @KUMAR Abhishek ,

I have communicated with @ChiragMishraMSFT-1092 about this and we have verified with ADF engineering team and confirmed that the requirement you are trying to achieve is not supported.

We request you to please provide your suggestion/feedback in ADF feedback forum and do share the feedback link here, so that other users with similar idea/suggestion can up-vote and/or comment on your feedback to increase the priority of the feature request.

ADF User voice forum: https://feedback.azure.com/forums/270578-azure-data-factory

All the feedback shared in this forum are actively monitored and reviewed by ADF engineering team.

----------

Thank you

Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

Answer 2

@KranthiPakala-MSFT

I have shared the feedback on ADF feedback forum
and posted new idea for this support:

https://feedback.azure.com/forums/270578-data-factory/suggestions/40594570-support-for-dynamic-content-with-pagination-rule-i