question

AndrewBatchelor-3038 avatar image
0 Votes"
AndrewBatchelor-3038 asked AndrewBatchelor-3038 commented

Gateway TImeout on auditLogs/signIns

While calling the auditLogs/signIns endpoint using paging via $top and $skiptoken, after retrieving dozens of pages sucessfully, some applications will return a 504 gateway timeout error.

Once we get this error, attempting to retry with the same url, we get a 400 with "Skip Token is null" while when we get a 429 error, we are able to retry with the same url after the throttling period has passed.

We have tried using smaller page sizes (as low as 500 records) but still run into the same problem for the same applications.

any idea what could be causing this issue?

Perhaps a certain signin record has more data than others and leads to a timeout?

Also is there a way to limit the data returned per sign in, all we really need is the date of the signin and a user id, but the $select option is not supported.

Here are the headers and response from the Gateway Time Error:
URL: https://graph.microsoft.com/beta/auditLogs/signIns?$filter=createdDateTime+ge+2022-03-16+and+appId+eq+%<<appid>>%27&$top=500&$skiptoken=47d96df265450ef838de40dac783215e22736d0aede079fb5f631fe09364a515

"request-id" "de8908f3-f79e-4f1f-b672-6977089b9329"
client-request-id "de8908f3-f79e-4f1f-b672-6977089b9329"
"x-ms-ags-diagnostic" {"ServerInfo":{"DataCenter":"West US 2","Slice":"E","Ring":"1","ScaleUnit":"002","RoleInstance":"MWH0EPF0005A6AE"}}
"Date" Fri, 15 Apr 2022 03:29:02 GMT

Response Body:
{"error":{"code":"UnknownError","message":"","innerError":{"date":"2022-04-15T03:29:02","request-id":"de8908f3-f79e-4f1f-b672-6977089b9329","client-request-id":"de8908f3-f79e-4f1f-b672-6977089b9329"}}}


And Then if we retry with the same URL:
{"error":{"code":"UnknownError","message":"Invalid Skip Token, skip token is null","innerError":{"date":"2022-04-15T03:36:00","request-id":"c48d64c7-8cd7-487a-901e-b06df5090161","client-request-id":"c48d64c7-8cd7-487a-901e-b06df5090161"}}}

This was on the beta API but experienced the same behavior on the v1.0 version as well

We do use the retry-after header and back off calls for the interval given. It doesn't seem like it's an throttling issue so much as a problem with a specific page load.

azure-ad-audit-logsazure-ad-sign-in-logs
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Could you please provide request id and timestamp of the error?

0 Votes 0 ·
AndrewBatchelor-3038 avatar image AndrewBatchelor-3038 VickyKumarMindtreeConsultingPVTLTD-5545 ·

Added the additional information in the question, thanks!

0 Votes 0 ·

1 Answer

VickyKumarMindtreeConsultingPVTLTD-5545 avatar image
0 Votes"
VickyKumarMindtreeConsultingPVTLTD-5545 answered AndrewBatchelor-3038 commented

Throttling behavior can depend on the type and number of requests. For example, if you have a high volume of requests, all request types are throttled. there might be case if you have consumed more than 0.8 of its limits, its mentioned in docs.

193016-image.png

The following are best practices for handling throttling:

Reduce the number of operations per request.
Reduce the frequency of calls.
Avoid immediate retries, because all requests accrue against your usage limits.

Note : there are also a Service-specific limits

for info about throttling please take a look on doc -https://docs.microsoft.com/en-us/graph/throttling



image.png (45.5 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi AndrewBatchelor-3038 ,

Have got time to check the given docs? Please let me know if you have any questions and I can help you further!

If this answer helped you, please mark it as "Verified" so other users can reference it.

0 Votes 0 ·
AndrewBatchelor-3038 avatar image AndrewBatchelor-3038 VickyKumarMindtreeConsultingPVTLTD-5545 ·

We've implemented retry-after (and also add progressive backoff on top of that) I did not receive the x-ms-resource-unit or x-ms-throttle-limit-percentage headers in any of the responses. We are able to pull 50+ pages of sizes ranging from 100 - 1000 but seem to get the 504 error on what could be a specific record. We use the same logic successfully on hundreds of different applications within the same tenant, and across dozens of other tenants.

I've opened a ticket via our Azure subscription support so we can hopefully get more feedback on what the specific error is.

0 Votes 0 ·