Graph API - Returning 404 in certain data centers when accessing on-prem mailboxes

DePalma, Mark 1

I recently created a process that reads/writes contacts in users' mailboxes on-prem via Graph. This process works as expected most of the time but certain times it fails with a 404. I've narrowed it down to the datacenter Graph is using. I've verified nothing is causing this issue at the network perimeter and no API calls are actually reaching the Exchange servers when the 404 is returned.

404 - North Central US

DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40fomain.com/contacts?$top=100
HTTP/1.1 404 Not Found
request-id: f245c3b7-4852-4494-aaf9-1424bfd9e41c
client-request-id: f245c3b7-4852-4494-aaf9-1424bfd9e41c
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Central
US","Slice":"SliceC","Ring":"2","ScaleUnit":"000","RoleInstance":"AGSFE_IN_56"}}
Strict-Transport-Security: max-age=31536000
Cache-Control: private

200 - East US

DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40domain.com/contacts?$top=100
HTTP/1.1 200 OK
Vary: Accept-Encoding
request-id: 2d014679-7c0c-4a68-bb3a-26e7cf91af53
client-request-id: 2d014679-7c0c-4a68-bb3a-26e7cf91af53
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"East
US","Slice":"SliceC","Ring":"5","ScaleUnit":"003","RoleInstance":"AGSFE_IN_41"}}
OData-Version: 4.0
Strict-Transport-Security: max-age=31536000
Cache-Control: private
Date: Wed, 25 Nov 2020 20:31:56 GMT

404 - North Europe

DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40domain.com/contacts?$top=100
HTTP/1.1 404 Not Found
request-id: 4ddb30db-e9f7-4a20-bd46-d58d9226fc0b
client-request-id: 4ddb30db-e9f7-4a20-bd46-d58d9226fc0b
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North
Europe","Slice":"SliceC","Ring":"4","ScaleUnit":"005","RoleInstance":"AGSFE_IN_2"}}
Strict-Transport-Security: max-age=31536000
Cache-Control: private
Date: Wed, 25 Nov 2020 21:52:06 GMT

Deva-MSFT 2,256 Reputation points Microsoft Employee

2020-11-26T05:05:41.95+00:00
Can you repro the issue without $top or not?

Is the issue intermittent or you can repro at will?

Try testing outside of your application using Microsoft Graph Explorer or POSTMAN and see if you can repro the issue for these users/mailboxes.
DePalma, Mark 1 Reputation point

2020-11-26T14:44:31.533+00:00
Without pagination I still receive the error.

DEBUG: GET https://graph.microsoft.com/v1.0/users/user%40domain.com/contacts HTTP/1.1 404 Not Found request-id: f7386d40-7d84-4d7c-8e2f-429eeb18780f client-request-id: f7386d40-7d84-4d7c-8e2f-429eeb18780f x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Europe","Slice":"SliceC","Ring":"4","ScaleUnit":"001","RoleInstance":"AGSFE_IN_69"}} Strict-Transport-Security: max-age=31536000 Cache-Control: private Date: Thu, 26 Nov 2020 14:29:20 GMT

I can consistently repro the issue when initiating from our London location since that hits the 'North Europe' datacenter for Graph. Like I said originally, this is consistent with certain datacenters.

I haven't done it in Graph Explorer yet, but it should happen there to if Graph Explorer uses one of the problematic datacenters.
Deva-MSFT 2,256 Reputation points Microsoft Employee

2020-11-26T18:15:21.577+00:00

Thanks for the update. Please note that you're trying to use Graph API with Exchange Hybrid deployment and it's in preview.
DePalma, Mark 1 Reputation point

2020-11-26T21:13:13.623+00:00

Case has been logged. I referenced you and the stackoverflow thread. They are moving forward with the ticket so far.