Window server time 2016 out of sync on particular timing

Question

question

kluangguy-3237 asked · Mar 02 at 11:03 AM

Window server time 2016 out of sync on particular timing

Dear all,
We have a AD that serves as time source for the domain member. The AD is synching the time from our internal NTP server. We noticed all the server time (AD server, and domain member) is auto adjusted with 8 hours ahead (from event log, the source is from Kernel-General) daily around 4 pm. After 1 hour, the server time is auto adjusted back.

1) How can we derive which application/system component adjusted the timing? as per the event log, we cannot identify.
2) what cause the server timing auto adjusted on certain timing? how can we prevent it?

all servers are running Windows server 2016, OS Build 14393.2906

thanks

From Domain Member
w32tm /query /status
Leap Indicator: 0(no warning)
Stratum: 6 (secondary reference - syncd by (S)NTP)
Precision: -6 (15.625ms per tick)
Root Delay: 0.0786569s
Root Dispersion: 0.2389291s
ReferenceId: 0xC0A8640D (source IP: 192.168.100.13)
Last Successful Sync Time: 3/2/2021 6:48:13 PM
Source: DxxxAD01.xxxx.sg
Poll Interval: 9 (512s)

From AD
w32tm /query /status

Leap Indicator: 0(no warning)
Stratum: 5 (secondary reference - syncd by (S)NTP)
Precision: -6 (15.625ms per tick)
Root Delay: 0.0801682s
Root Dispersion: 0.1626273s
ReferenceId: 0x0A4681BE (source IP: 10.xx.xx.190)
Last Successful Sync Time: 2/3/2021 6:48:04 PM
Source: 10.xx.xx.190,0x8
Poll Interval: 10 (1024s)

windows-server-2016

timeautoadjusted.jpg (99.3 KiB)

· 1

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DaisyZhou-MSFT · Mar 12 at 08:52 AM

Hello @kluangguy-3237,
How are things going on your end? Please keep me posted on this issue.
If you have any further questions or concerns about this question, please let us know.
I appreciate your time and efforts.

Best Regards,
Daisy Zhou

0 ·

Answer 1 · 2021-03-15T11:00:52.853Z

kluangguy-3237 answered · Mar 15 at 11:00 AM

Hi all,
the issue is related to VMWare is synching time to the VM. After applying the configuration (https://kb.vmware.com/s/article/1189), the out of sync issue is resolved

Thanks @DaisyZhou-MSFT sharing the guide on narrow down the issue. thanks @DSPatrick input :)

·

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 2 · 2021-03-02T15:24:27.733Z

DSPatrick answered · Mar 02 at 03:24 PM

Kernel-General event ID 1 occurs whenever Windows changes the system time. Windows changes the system time whenever it detects that the authoritative time differs from the system clock on that server so you may have an issue with your NTP source.

--please don't forget to Accept as answer if the reply is helpful--

·

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 3 · 2021-03-03T03:34:22.977Z

DaisyZhou-MSFT answered · Mar 03 at 03:34 AM

Hello @kluangguy-3237,

Thank you for posting here.

To better understand our question, please confim the following information at your convenience:
1.Is your PDC physical machine or virtual machine? Or is your PDC hosted on Vmware/Hyper-V?
2.Is your AD environment single forest with single domain?
3.How many DCs in your Domain? Are they physical machine or virtual machine?
4.Did this problem happen suddenly? Or did we make any changes before the problem occurred?
5.Based on "We noticed all the server time (AD server, and domain member)", do you mean time on all the machines including all DCs and all workstations(member servers and domain clients) is auto adjusted with 8 hours ahead?
6.When the issue occurs, does the time on your internal NTP server normal?

Meanwhile, please check if the time configurations on all the machines are correct.

===PDC===

HLM\SYSTEM\CurrentControlSet\services\w32time\TimeProviders\VMICTimeProvider
Name: Enabled
Type: REG_DWORD
Data:0

Only if your PDC is virtual machine, you need to set the first entry.

Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config
Key Name: AnnounceFlags
Type: REG_DWORD (DWORD Value )
Data: 0x5

Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\Type
Key Name: Type
Type: REG_SZ(String Value)
Data: NTP

Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters
Key Name: NtpServer
Type: REG_SZ(String Value)
Data: time.windows.com,0x9

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer
Key Name: Enabled
Type: REG_DWORD
Data: 1

===other DCs & domain clients & member servers===
Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\Type
Key Name: Type
Type: REG_SZ(String Value)
Data: NT5DS

Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config
Key Name: AnnounceFlags
Type: REG_DWORD (DWORD Value )
Data: 0xa

If time configurations are OK, we can configure audit policy for one/some/all the server that time (AD server, and domain member) is auto adjusted with 8 hours ahead.

Create GPO and link it to the OU with machines above and edit GPO as below:

Legacy audit policy:
Computer Configuration\Windows settings\security settings\local policies\audit policy\audit system events – Success and Failure

Or use advanced audit policies
(Tip: by default, once any advanced audit policy setting is configured, audit policies will overwrite all Legacy audit policies,so if you have not configured any advanced audit policy setting so far, please configure Legacy audit policy ):
Computer Configuration\Windows settings\security settings\Advanced Audit Policy Configuration\System\Audit Security State Change – Success and Failure

We can run the following commands on the domain controller to force the refresh policy and check whether the related audit policy settings are enabled:

gpupdate /force
auditpol /get /category:*

Then if the issue reoccurs, we can check the event ID 4616 via Security log\Event Viewer, check if there is any information we can get.

Reference
4616(S): The system time was changed.
https://docs.microsoft.com/en-us/windows/security/threat-protection/auditing/event-4616

Hope the information above is helpful.

Should you have any question or concern, please feel free to let us know.

Best Regards,
Daisy Zhou

· 4 ·

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

kluangguy-3237 · Mar 04 at 02:47 AM

Hi Daisy,
thanks for your response, i have provided more detail on my enviroment

1.The PrimaryDomainController is running as VM (VMWare)

2.there is two AD server (each with one domain) and they formed a forest trust

3.there is one DC in the domain which is the PDC and it running as VM

4.It happened for a while till we noticed some of the application stop working. the timing of the server has some issue when we get handed over

5.Yes, the DC, and the members are auto-adjusted with 8 hours ahead

do not have access, however other machine (diff domain) that sync from the same ntp is fine

For the PDC registry
a) VMICTimeProvider
Name: Enabled
the current value is 1, should i change it to 0?

b) W32Time\Config
Key Name: AnnounceFlags
the current value is 5

c) W32Time\Parameters
Key Name: NtpServer
the value is pointing to two internal NTP server (10.xx.xx.xx0,0x8 10.xx.xx.x3,0x8)

For one of the domain DC member
a) W32Time\Config
Key Name: AnnounceFlags
current value is a

0 ·

DaisyZhou-MSFT kluangguy-3237 · Mar 10 at 09:32 AM

Hello @kluangguy-3237,

I am sorry for the late reply.

Thank you for your update.

Yes, we can check the configuration above I provided on your PDC and non-PDC.

Have you turn off the time synchronization in VMWare? I means we do not let time on VM PDC synchronize with VMWare.

Best Regards,
Daisy Zhou

0 ·

kluangguy-3237 DaisyZhou-MSFT · Mar 12 at 02:39 PM

From the security event 4616, i can see the vmtoolsd.exe is the one changing the timing; however when checking the status (timesync status) it is already disable as below image

i am rechecking the VM setting of the VM, I hope this is the culprit. will share if i have good news :)

0 ·

securityauditlog-timesync.jpg (144.3 KiB)

Show more comments

Answer 4 · 2021-03-15T13:07:29.337Z

DSPatrick answered · Mar 15 at 01:07 PM

Sounds good, you're welcome.

--please don't forget to Accept as answer if the reply is helpful--

·

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

question