Back to HPET-TSC timers or why underclocking CPU helps

Everything about latency. Tips, testing methods, mouse lag, display lag, game engine lag, network lag, whole input lag chain, VSYNC OFF vs VSYNC ON, and more! Input Lag Articles on Blur Busters.
triplese
Posts: 130
Joined: 13 Dec 2021, 12:20

Back to HPET-TSC timers or why underclocking CPU helps

Post by triplese » 27 Oct 2022, 06:06

On my work I have a big problem with my prod environment - all VMs on hypervisor got massive time drift.
BIOS was configured as HPET on, windows bcdedit - default.
In eventlog I see

Code: Select all

Event ID: 50
Source: W32time
Type: Warning
The time service detected a time difference of greater than 5000 milliseconds for 900 seconds. The time difference might be caused by synchronization with low-accuracy time sources or by suboptimal network conditions. The time service is no longer synchronized and cannot provide the time to other clients or update the system clock. When a valid time stamp is received from a time service provider, the time service will correct itself. 
VMs getting time from NTP, but it keeps drifting after sync.
I started investigating this and found some interesting info, where from 2010 was big problems with SQL server and hypervisors and many windows KB and whitepapers was published about it. I tried to squeeze it:
CPU has timers for each core. Manufacturer doesnt guarantee sync between it, so for accurate timings system need to use HPET.
However, using high-frequency OS timer (1ms as it does steam on launching itself) with unstable per-core TSC timer causing so-called desync, or time drift.
And now we have interesting point - why UNDERclocking CPU helps few people here to get 'stablish' gaming - more cores and higher their frequency causing bigger drift, and any dynamic power management (turboboost, etc) causing even more drift.
After fixing it on prod I tried on my home PC with 16-core 5950x to turn off HPET in bios, set constant CPU voltage, frequency, FSB and disable spread spectrum to fix values as much constant as possible. And looks like it fixed my kb-mouse desync - I can bhop now with mousewheel even on 64 tick server, even not local! Hitreg seems slightly improved, but needs more placebo testing.

However, this is a duct-tape "fix" and needs more deep investigation - why HPET isnt working correctly (BIOS bug?), is it HPET+TSC desync issue or steam bug.

triplese
Posts: 130
Joined: 13 Dec 2021, 12:20

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by triplese » 27 Oct 2022, 06:07

We can try to collect list of affected systems (CPU+MB, bios ver) to take some stats.
So I will start and add here:

AFFECTED:

Code: Select all

5950x, gigabyte b550 aorus pro v2, all bios ver
5800x Gigabyte X570 Aorus Elite, all Versions.
5800x, ASUS TUF X570-PLUS, all BIOS ver.
5600x, asus rog strix x570 f-gaming
UNAFFECTED:

Code: Select all



Last edited by triplese on 28 Oct 2022, 04:50, edited 4 times in total.

Anonymous768119

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by Anonymous768119 » 27 Oct 2022, 06:45

triplese wrote:
27 Oct 2022, 06:06
On my work I have a big problem with my prod environment - all VMs on hypervisor got massive time drift.
BIOS was configured as HPET on, windows bcdedit - default.
In eventlog I see

Code: Select all

Event ID: 50
Source: W32time
Type: Warning
The time service detected a time difference of greater than 5000 milliseconds for 900 seconds. The time difference might be caused by synchronization with low-accuracy time sources or by suboptimal network conditions. The time service is no longer synchronized and cannot provide the time to other clients or update the system clock. When a valid time stamp is received from a time service provider, the time service will correct itself. 
VMs getting time from NTP, but it keeps drifting after sync.
I started investigating this and found some interesting info, where from 2010 was big problems with SQL server and hypervisors and many windows KB and whitepapers was published about it. I tried to squeeze it:
CPU has timers for each core. Manufacturer doesnt guarantee sync between it, so for accurate timings system need to use HPET.
However, using high-frequency OS timer (1ms as it does steam on launching itself) with unstable per-core TSC timer causing so-called desync, or time drift.
And now we have interesting point - why UNDERclocking CPU helps few people here to get 'stablish' gaming - more cores and higher their frequency causing bigger drift, and any dynamic power management (turboboost, etc) causing even more drift.
After fixing it on prod I tried on my home PC with 16-core 5950x to turn off HPET in bios, set constant CPU voltage, frequency, FSB and disable spread spectrum to fix values as much constant as possible. And looks like it fixed my kb-mouse desync - I can bhop now with mousewheel even on 64 tick server, even not local! Hitreg seems slightly improved, but needs more placebo testing.

However, this is a duct-tape "fix" and needs more deep investigation - why HPET isnt working correctly (BIOS bug?), is it HPET+TSC desync issue or steam bug.
I have something similar. When I checked my UDP latency to VPS server I got 500-1100ms which means server time and my PC time were not in sync. I was able to fix it temporarily by manually setting the time ie. advancing it or retarding for 1 second and then putting it back to auto mode with time.google.com time server. After that latency was around 11-40 ms but gradually increased with every hour. Something is definitely wrong here but the question is if it's about oscillators or/and internet connection.

triplese
Posts: 130
Joined: 13 Dec 2021, 12:20

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by triplese » 27 Oct 2022, 08:18

a_c_r_e_a_l wrote:
27 Oct 2022, 06:45
When I checked my UDP latency to VPS server I got 500-1100ms which means server time and my PC time were not in sync. I was able to fix it temporarily by manually setting the time ie. advancing it or retarding for 1 second and then putting it back to auto mode with time.google.com time server. After that latency was around 11-40 ms but gradually increased with every hour. Something is definitely wrong here but the question is if it's about oscillators or/and internet connection.
IMHO adding another layer (in this case BIG layer - internet) with unknown routing and possible congestion is not way to resolve problem, try to divide et impera :)
If you have your local network - you can try to do this one more time.
My problem was that I have "heavy" model - cant jump in proper tick, like my mouse not in sync with system itself, so when I rotating wheel to jump, events from wheel is delayed.

Another possible problem - HPET is misreporting its frequency, so when I turned it off, my okay-ish TSC is better syncing.

Anonymous768119

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by Anonymous768119 » 27 Oct 2022, 09:11

triplese wrote:
27 Oct 2022, 08:18
a_c_r_e_a_l wrote:
27 Oct 2022, 06:45
When I checked my UDP latency to VPS server I got 500-1100ms which means server time and my PC time were not in sync. I was able to fix it temporarily by manually setting the time ie. advancing it or retarding for 1 second and then putting it back to auto mode with time.google.com time server. After that latency was around 11-40 ms but gradually increased with every hour. Something is definitely wrong here but the question is if it's about oscillators or/and internet connection.
IMHO adding another layer (in this case BIG layer - internet) with unknown routing and possible congestion is not way to resolve problem, try to divide et impera :)
If you have your local network - you can try to do this one more time.
My problem was that I have "heavy" model - cant jump in proper tick, like my mouse not in sync with system itself, so when I rotating wheel to jump, events from wheel is delayed.

Another possible problem - HPET is misreporting its frequency, so when I turned it off, my okay-ish TSC is better syncing.
Well yes, routing for UDP is unknown, when playing on brutalcs servers I feel like the issue is 2 times worse however my ping does not exceed 60 ms. I tried to understand how clock sync works but it is beyond my abilities and the topic looks very complicated. I can understand that some additional latencies can appear due to EMI but I would say it's rather about nanoseconds or few miliseconds instead of 200-300 ms added to every game I play. That would suggest some serious problems in keeping time in sync.

espresso
Posts: 63
Joined: 11 Feb 2022, 04:56

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by espresso » 27 Oct 2022, 15:16

triplese wrote:
27 Oct 2022, 06:07
We can try to collect list of affected systems (CPU+MB, bios ver) to take some stats.
So I will start and add here:

AFFECTED:

Code: Select all

5950x, gigabyte b550 aorus pro v2, all bios ver

This is a good Idea and probably should have been done extensively long time ago.
As a matter of fact it should be mandatory to post Hardware ID when creating new post.

Code: Select all

5800x Gigabyte X570 Aorus Elite, all Versions.

Anonymous768119

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by Anonymous768119 » 27 Oct 2022, 15:47

Code: Select all

5800x, ASUS TUF X570-PLUS, all BIOS ver.

MegaMelmek
Posts: 235
Joined: 21 Jan 2021, 12:54

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by MegaMelmek » 27 Oct 2022, 16:03

5600x, asus rog strix x570 f-gaming ( i have tested many cpus and mobos they are all same for me)
Downclock helps me but the iron is still the best.
My explanation is if you downclock cpu and dissable all kind of stuf your PC consume less power so still can be power related…

MegaMelmek
Posts: 235
Joined: 21 Jan 2021, 12:54

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by MegaMelmek » 27 Oct 2022, 16:12

a_c_r_e_a_l wrote:
27 Oct 2022, 09:11
triplese wrote:
27 Oct 2022, 08:18
a_c_r_e_a_l wrote:
27 Oct 2022, 06:45
When I checked my UDP latency to VPS server I got 500-1100ms which means server time and my PC time were not in sync. I was able to fix it temporarily by manually setting the time ie. advancing it or retarding for 1 second and then putting it back to auto mode with time.google.com time server. After that latency was around 11-40 ms but gradually increased with every hour. Something is definitely wrong here but the question is if it's about oscillators or/and internet connection.
IMHO adding another layer (in this case BIG layer - internet) with unknown routing and possible congestion is not way to resolve problem, try to divide et impera :)
If you have your local network - you can try to do this one more time.
My problem was that I have "heavy" model - cant jump in proper tick, like my mouse not in sync with system itself, so when I rotating wheel to jump, events from wheel is delayed.

Another possible problem - HPET is misreporting its frequency, so when I turned it off, my okay-ish TSC is better syncing.
Well yes, routing for UDP is unknown, when playing on brutalcs servers I feel like the issue is 2 times worse however my ping does not exceed 60 ms. I tried to understand how clock sync works but it is beyond my abilities and the topic looks very complicated. I can understand that some additional latencies can appear due to EMI but I would say it's rather about nanoseconds or few miliseconds instead of 200-300 ms added to every game I play. That would suggest some serious problems in keeping time in sync.
my ping not go over 5ms still is unplayable so this is not internet issue for most of the ppl… strange is that most off the ppl with this issue hase ryzen CPU now so maybe is time give intel a try again… i have i7 7700 and that was no issue there i do not even know what input lag was since a sell it i was trying 9700kf 5600x 5800x 2600x 1600x 10400 all same all on different mobo

Anonymous768119

Re: Back to HPET-TSC timers or why underclocking CPU helps

Post by Anonymous768119 » 27 Oct 2022, 16:19

Majority use Intel CPUs, so far I didn't try it but I don't think it's related.

Post Reply