The Windows Server Essentials Service cannot synchronize the user password to the online identity repository

Jesse Flintoff 216

Hi there.

Recently. I have the below error coming up on multiple 2016 essential servers when doing password changes via the Essentials Dashboards.

I can confirm that passwords are not syncing to 365.

I have tired disabling 365 integration. Then re enabling. Same issue.

Anyone else seen this issue?

STM 1 Reputation point

2021-07-23T08:32:27.86+00:00

... any news?
Franck Gravel 1 Reputation point

2021-07-26T21:36:36.637+00:00

Same here and My users are freaking out! please keep me updated
Manson DeGraw 1 Reputation point

2021-07-29T17:21:15.057+00:00

We are also having this same issue with all clients using Server 2016 Essentials 2016 standard with Essentials Role installed. Password sync to Office 365 stopped working recently. Have not checked clients with Server 2012 R2 Essentials but will check them. I seems very odd that creating a new user with sync to Office 365 and create the password correctly In process of opening a MS ticket.

Manson
Shawn P. Lemay 21 Reputation points

2021-07-31T17:48:50.98+00:00

So glad I found this thread, I was just about to open a case to Microsoft, as I have 3 sites now failing with this error. Hopefully, they get this fixed soon!
Phillip Fielding 1 Reputation point

2021-08-06T09:22:11.563+00:00

Like others, ive now got 4 different customers experiencing this issue.
this is the only thread ive managed to find with any information.
has there been any update on this?
MAC 1 Reputation point

2021-08-06T20:17:26.877+00:00

I am also experiencing this issue. Any update?
Zachary Sigamony 1 Reputation point

2021-09-27T17:53:04.68+00:00

I have a similar issue too with two of my clients on server 2016 essential
Zachary Sigamony 1 Reputation point

2021-09-27T17:53:13.49+00:00

I have a similar issue too with two of my clients on server 2016 essential
Ishtiaq 71 Reputation points

2021-10-08T14:00:25.907+00:00

Good news, Password synch is now working, although I've not yet heard it from Microsoft in response to our support ticket. Just tested on a couple of Client server and it works!
Lets hope its a permanent fix...

45 answers

Linus Haake 31 Reputation points

2021-09-15T18:00:40.68+00:00

I've tested it here, the content did not seem to be trash, at least from what I could see.
However, I've replaced the file from the backup (31st of May, since some folks meant that it already happened in June 21) and changed the password of a test-user, but it still does not seem to be updated for the related object in the tenant.
I will give it a while, but MS told me that the issue is not with the local server, but with some changes on Microsoft Graph (Azure).
Please sign in to rate this answer.
Linus Haake 31 Reputation points

2021-09-15T19:57:34.357+00:00

Did not help at all, neither the issue is resolved by workaround, nor the error messages did not re-appear again after I've deleted them.
Trying to logon with the password changed before does also not work.

joe mills 26 Reputation points

2021-09-16T15:36:34.713+00:00

I am very sorry to hear that the workaround is not working in your particular situation. I find it odd that your config files are NOT being corrupted by the MS backend problem. I am curious if your Cloud folder also contains a "CloudServiceEnvironment-XX.xml" file and if it is not corrupt.

Linus Haake 31 Reputation points

2021-09-16T15:47:18.593+00:00

It exists but does not look corrupt at all.
Sign in to comment
TimRaines 6 Reputation points

2021-09-16T02:11:28.417+00:00

Hi All,

I have been watching this thread for sometime now and ended up also deciding to log support cases with Microsoft across three different customers we support, who are also facing the same issues. My cases are with the Partner support team at the moment trying to get them re-routed has been very difficult and even referencing this thread has still made it challenging to get the support engineers to agree its a Graph API issue not an on-premise server issue. In the last day I think I have made some progress with them however for those watching this thread; I just wanted to provide some further details on my findings.

The log file for the Essentials dashboard consistently records the error on each effected customers server the same. After a password reset is performed via the Dashboard; the following exception is logged. This supports the findings so far in this thread that the Graph API endpoint is broken. The 404 error is clear that the process is trying to hit a non-existent endpoint.

Log File Path: C:\ProgramData\Microsoft\Windows Server\Logs\Dashboard.log

[15504] 210914.233059.5603: O365Manager: TResult : Boolean, args : Microsoft.WindowsServerSolutions.Common.ProviderFramework.OperationInvokeEventArgs1[System.Boolean] [15504] 210914.233059.5603: O365Manager: actualArgs : Microsoft.WindowsServerSolutions.Common.ProviderFramework.OperationInvokeEventArgs1[System.Boolean]
[15504] 210914.233059.5743: OIMUtils: GetO365ConfigurationErrorMessage for Unexpected
[15504] 210914.233059.6323: OIMUserExtension2: Error occurred in [BeginChangePassword], Title : An unknown error occurred, Content: Make sure that the computer is connected to the Internet, and then try again.

If you continue to experience this problem, restart the server and then try again.
[4968] 210914.233059.6363: O365ResetPasswordDialogAddin: Failed to reset password : Microsoft.WindowsServerSolutions.Common.UserExtensionException: Failed to call Graph service with status code 404 and error: {"error":{"code":"ResourceNotFound","message":"Invalid version: v2","innerError":{"date":"2021-09-14T13:30:59","request-id":"cffb90cf-20af-40ac-91c8-49012f755eee","client-request-id":"cffb90cf-20af-40ac-91c8-49012f755eee"}}} ---> Microsoft.WindowsServerSolutions.O365Integration.O365ConfigureException: Failed to call Graph service with status code 404 and error: {"error":{"code":"ResourceNotFound","message":"Invalid version: v2","innerError":{"date":"2021-09-14T13:30:59","request-id":"cffb90cf-20af-40ac-91c8-49012f755eee","client-request-id":"cffb90cf-20af-40ac-91c8-49012f755eee"}}}
at Microsoft.WindowsServerSolutions.O365Integration.GraphAdapter.DefaultHandleGraphServiceError(Int32 statusCode, GraphServiceError error, Int32 retryTimes, Action2 errorHandler) at Microsoft.WindowsServerSolutions.O365Integration.GraphAdapter.<>c__DisplayClass60_11.<InvokeWebServiceWithRetry>b__1(Int32 statusCode, GraphServiceError error)
at Microsoft.WindowsServerSolutions.O365Integration.GraphAdapter.HandleGraphServiceException[TResult](Func1 func, Func3 errorHandler)
at Microsoft.WindowsServerSolutions.O365Integration.GraphAdapter.InvokeWebServiceWithRetry[TResponse](Func2 func, Action2 errorHandler)
at Microsoft.WindowsServerSolutions.O365Integration.O365ManagementCore.ChangePassword(String localUserName, String password)
at Microsoft.WindowsServerSolutions.O365Integration.O365ManagementProvider.<>c__DisplayClass30_0.<ChangePassword>b__0()
at Microsoft.WindowsServerSolutions.Common.ProviderFramework.ProviderBase1.InvokeOperation[TResult](String operationName, Func1 func)
--- End of inner exception stack trace ---
at Microsoft.WindowsServerSolutions.O365Integration.OIMUserExtension2.SyncCallWarpper(Action`2 action)
at Microsoft.WindowsServerSolutions.O365Integration.DashboardTabs.ResetPasswordDialogAdorner.<>c__DisplayClass2_0.<CreatePreExecutionTasks>b__0()
[23348] 210914.233100.7943: FormContentExtension: no tasks found.
[28032] 210914.233222.2273: TestOnlu: O365TabPageCtrlb1d8aaa7-0be9-43d1-a36e-4789f98ea013

Do others in this thread see this kind of error reported in their logs as well?
Please sign in to rate this answer.
Shawn P. Lemay 21 Reputation points

2021-09-16T16:07:35.647+00:00

Yes - I see similar messages - certainly the first few lines, but I did not see the failed to reset password (but I also didn't try and reset a password prior to checking the log).

Paul C 1 Reputation point

2021-09-21T13:06:21.427+00:00

Same messages here. It would be highly appreciated if Microsoft give us a solution
Sign in to comment
Linus Haake 31 Reputation points

2021-09-21T13:50:39.373+00:00

Microsoft asked me whether I would agree if they archive the ticket, because there is no internal response from the product group and they cannot tell me any estimation by when it will be solved.
I of course disagreed and to me it feels like they are not having enough tickets related to this issue...
Please sign in to rate this answer.
Matschek 1 Reputation point

2021-09-21T14:28:17.803+00:00

So it may be helpful if all the other ones watching this thread open a ticket on their own, and refer to your ticket? If you think so @Linus Haake , would you share the ticket id?

Linus Haake 31 Reputation points

2021-09-21T14:30:56.057+00:00

sure, it's [Case #:27076107] with Microsoft Office 365 Support, but there should be plenty of others...

Matschek 1 Reputation point

2021-09-21T15:55:04.6+00:00

""should be"... Don't underestimate how few people make the effort and really open a ticket. Two years ago I reported an issue with the German edition of the Essentials 2016 and it took quite a while until the product team confirmend the problem (Cannot edit security settings of a share with the dashboard). According to the supporter, nobody reported it before and they told me they won't fix it because I'm the only one who is interested in.

Rob Johns 1 Reputation point

2021-09-22T07:18:38.767+00:00

I'm experiencing this issue and will happily raise a ticket if someone tells me how. Never had to before, always found the answer on the internet because someone else had solved it. In fact i've just created my account so I can raise the issue.

Matschek 1 Reputation point

2021-09-22T10:21:41.74+00:00

Tickets can be opened here: https://serviceshub.microsoft.com/supportforbusiness/create
If you are a Microsoft Partner (Action Pack / Silver / Gold) you have some free tickets you can use here.
Don't know how it is progressed if no partnership is available and you choose to pay support, because bug support ist free if bug is confirmed.

I just opend a new ticket and referenced to the one given by Linus

TimRaines 6 Reputation points

2021-09-23T06:48:21.667+00:00

We are a Silver Partner and had a very difficult time trying to convince the Partner support team who are also the same people running M365 support that this was not a on-premise server issue. Finally now they realise that it's a problem with the Graph API and are awaiting updates from the backend product group - so most definitely the more cases logged on the issue will hopefully bring more attention to the problem for a fast fix.

These are the three cases I had opened with M365/Office 365 Support. I don't see much benefit using the Business Support link above since this is not a on-prem issue and the backend product group are well aware of the problems and have acknowledge to me and others in the thread that the fix is being worked on. I also got the same "no timeframe" answer which is concerning me a bit. Also what is concerning me most is that the Partner support team had pushed me to allow them to close all 3 cases I opened and wanted to provide updates with me directly when they heard more from the backend team. I fortunately have an open chat with one of the support team managers directly on Teams at the moment and he was encouraging me to check in with him regularly to get updates. So I will be chasing him up and will report back whatever new info I get.

These are the 3 case ID's I had opened and that are now closed on this issue:

27440182

27440193

27440195
Sign in to comment
Matschek 1 Reputation point

2021-09-22T10:54:32.02+00:00

I just got a call from MS and they confirmed that they have a lot of customers and a lot of tickets about this known issue.
Hotline still has no idea about the point of time when they are going to fix it. That's unbelievable.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
Richard Botfield 1 Reputation point

2021-09-22T11:38:54.297+00:00

I have just spoken with someone from the Microsoft 365 support team and they are suggesting that they would need to have a Gold Partner look into the issue further.

It seems like they are suggesting that this is an issue with our server configuration rather than Microsoft 365/Azure.
Please sign in to rate this answer.
TimRaines 6 Reputation points

2021-09-23T06:53:43.007+00:00

You don't need to talk to a Gold partner; this is a cloud issue and definitely not a server config issue. However if you are working with a MS Partner - gold or otherwise; make sure they are aware of the problem and ask them to also reach out to their Microsoft Support channels; as partners generally can push these issues further on behalf their customers.
Sign in to comment