Getting GraphApiJsonThrottlePerAppAndTenantThresholdExceeded Error in Data Factory

RAJANDINESHKUMAR-2744 1 Reputation point
2021-07-19T10:52:13.503+00:00

Hi Team,

I am reading Office 365 Data using graph api in Azure Data Factory. i am reading data in JSON format and stored in ADLS Gen 2.

i am getting this 'GraphApiJsonThrottlePerAppAndTenantThresholdExceeded' error while reading data. pasted the complete error message for reference. generating access token through WEB activity by passing Service principal. clientid etc.

In Copy Activity -- > created rest api linked service to read office data with Authorization header and passing value to the Pagination Rules parameter value as AbsoluteUrl: $['@odata.nextLink'].

i am getting the JSON file when the volume of JSON is less. i executed the same URL in PostMan by providing the access token, it shows the JSON data without any error.

Failure happened on 'Source' side. ErrorCode=RestSourceCallFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The HttpStatusCode 429 indicates failure. Request URL: https://graph.microsoft.com/beta/reports/getOffice365ActiveUserDetail(period='D7')?$format=application%2fjson&$top=1000&$skiptoken=MDoxNzc6ZXlKMGIydGxiaUk2SW Response payload:{"error":{"code":"UnknownError","message":"{\"error\":{\"code\":\"GraphApiJsonThrottlePerAppAndTenantThresholdExceeded\",\"message\":\"Please retry later\"}}","innerError":{"date":"2021-07-19T09:51:26","request-id":"a70da1c8-ec1d-4ff3-XXXX-a6a99cXXXXXX","client-request-id":""}}},Source=Microsoft.DataTransfer.ClientLibrary,'

could you please help us to resolve this. kindly provide if any other implementation approach is to be incorporated.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,487 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
    2021-07-22T09:39:40.75+00:00

    Hi @Anonymous ,

    Thank you for posting query in Microsoft Q&A Platform.

    When copying data from REST APIs, while to return large amount of data, API splits the result into multiple pages and requires callers to send consecutive requests to get next page of the result.

    Usually, the request for one page is dynamic and composed by the information returned from the response of previous page.

    Check below link to know more about how pagination works in REST connector in Azure data factory.
    https://learn.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support

    In above link example also next page URL was present in previous page response and there as using syntax as $.paging.next to invoke same as AbsoluteUrl. Kindly try to follow steps and details mentioned in above documentation link to resolve your issue.

    Hope this will help. Thank you.

    -----------------------

    • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification.