question

KothaiRamanathan-7384 avatar image
0 Votes"
KothaiRamanathan-7384 asked SaurabhSharma-msft commented

Rate limit exceeded Error - When creating a pipeline run from LA using azuredatafactory connection

I have a "ApiConnection" action in Logic App (using azuredatafactory connector), where I create a pipeline run in a data factory. However, frequently, I see this error message :

{
"message": "Rate limit is exceeded, clientRequestId: xxxx"
}

  1. Is this error occurring because the ADF connector is not able to handle the load, or is it that the data factory is not able to handle the load ?

  2. do suggest recommended actions to take in this scenario.

Thanks,
Kothai.



azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SaurabhSharma-msft avatar image
0 Votes"
SaurabhSharma-msft answered SaurabhSharma-msft commented

@kothairamanathan-7384 Thanks for using Microsoft Q&A !!
You could be getting this error as each connector in logic app has its own API call limits which would result in following errors. For ADF connector, throttling limits are defined as 100 calls per minute per connection. Refer to the Throttling Limits for ADF connector.
78170-image.png

Common suggestions are:
- Use separate connection for each logic app even if the underlying resource is same.
- Even within same logic app, use separate connection for different type of actions on the same resource.
- Instead calling action in for-each loop, possibly call child Logic Apps and implement action call there.
- While fetching large amount of data, use Pagination
- If Pagination is not available, create a logic to pull data in appropriate batches.
- Implement appropriate delays in-between.
- Despite all care taken, implement proper error handling and retry logic to handle 429 event
Handle errors and exceptions in Azure Logic Apps

Also, if usage requirement is really high or mission critical , you can go for ISE – fixed cost model where most of the limits are removed or uplifted. As it is an isolated environment, user can predict and control the load.



Please let me know if you have any other questions.


image.png (10.5 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@kothairamanathan-7384 Following up as I haven't heard back from you. Did my answer solve your issue? If so, please mark as accepted answer. If not, please let me know how I may better assist.

0 Votes 0 ·

@kothairamanathan-7384 Please let me know if you have any other questions.

0 Votes 0 ·