namcost, there are some big problems with test methodology, but your post is a bit over the top disrespectful...
Reviewers are not perfect.
There are problems with test methodology by all reviewers, but you have to understand it's possible to have a real-world 0.5ms lag
AND a real-world 6ms input lag simultaneously.
It's simply different stopwatching methodologies, since
not all pixels refresh at the same time. The pixels at the top edge can have 0ms lag, and the pixels at the bottom edge can have 16ms lag.
A panel can have 0.5ms lag during VSYNC OFF but 6ms lag during VSYNC ON.
It's a matter of how you start the lag stopwatch:
- Do they use a full-screen camera (first-any-pixel lag) or a single-point measurement (top, center, bottom)
- Do they use only Present(), or do you use Present()+Flush() as the timestamp for start?
- Do they use RasterStatus.InVBlank as the timestamp for start?
- Do they use a VSYNC detector on the video cable as the timestamp for start, or do you use the API on WIndows side? (video port codec delays, etc)
- What sync technology do they use? (see below)
And how you stop the lag stopwatch:
- Do they detect the pixels via a photodiode? Other method?
- Which GtG% do they trigger at? Pixels don't change color instantly, they "fade from one color to the other".
- Some reviewers stopwatch-stop at GtG2%, others at GtG10%, and yet others at GtG50%, yet others at GtG100%
- What temperature was lag measurements done at? (temperatures shifts the GtG curve, as colder LCDs have slower GtG)
- What color is the lag measured on? GtG's of different colors are different, so you get different lag numbers.
- Remember: Human reacts to pixels before they're GtG 100%. if you're trying to publish numbers matching human reaction time, you cannot lagtest to GtG100%!
And how the sync/strobe technology is configured
(Important: Do 100-1000 test passes each for TOP, and for CENTER, and for BOTTOM, and average the lag)
- VSYNC ON + nonstrobed means
TOP < CENTER < BOTTOM
- VSYNC OFF + nonstrobed means
TOP == CENTER == BOTTOM
- VSYNC ON + strobed (0 crosstalk, or crosstalk below GtG% stopwatch threshold) means
TOP == CENTER == BOTTOM
- VSYNC OFF + strobed (0 crosstalk, or crosstalk below GtG% stopwatch threshold) means
TOP > CENTER > BOTTOM
- strobed (bad crosstalk) sometimes goes weird like
TOP < CENTER > BOTTOM or
TOP > CENTER < BOTTOM
Certainly, some reviewers are often influenced by manufacturers to choose a methodology that favors specific conditions (e.g. VSYNC OFF esports lag with early GtG cutoff thresholds). They certainly get free samples from manufacturers, but obviously there are additional considerations at play.
Great napkin exercise conundrum: You know, the honorable GtG100% stopwatch end for lag testing is well-intended but totally misguided when GtG50% for a black->white transition means a pixel is already emitting half the light of a fullwhite -- and that's easy for eyes to already see. So a GtG50% lag test is much more human-reaction-time accurate than GtG100%, but it's hard to scientifically determine the exact threshold, as different human clicks -- some will see the GtG much sooner (e.g. by GtG10%) while others a bit later. Split the difference, GtG50% is a more noble lag-test compromise, but it is "influenced" by the need to match human reaction behaviours, rather than isolate to display-processing behaviours (GtG 2% is more representative of that, because once GtG has already started, we know the display has already "processed" the pixel). So one asks oneself -- is the goal to isolate lag to display, or try to match lag numbers to the approximate trigger point of a human reaction time? So that's why I'm no fan of GtG100% for latency tests. Now you see the conundrum!
But I am a messenger to inform you that I've seen "0.5ms" and "6ms" from exactly the same display, depending on how the above is tweaked.
As so many of us know, many 240Hz monitors have bad lag at 60Hz, so using a console-optimized 60Hz VSYNC ON lag tester like Leo Bodnar will produce more useful numbers relevant to console users than for PC-esports users (like RTINGS VSYNC OFF lag tester).
It's fine to criticize reviewer methodology, but I feel that this is beyond bounds of Blur Busters decorum. Ideally, please
criticize the chosen methodology is our mantra, rather than venting anger at the reviewer or manufacturer.
I definitely have a legitimate complaint too: Not all reviewer websites publish their stopwatch-start and stopwatch-stop methodology for their lag tests. And indeed, methodologies can be flawed and sometimes bad thresholds that diverge from human thresholds can be chosen, but you have to realize this reality.
We are considering a campaign "
Blur Busters Reviewer Test Methdology Disclosure Best Practices" to solve this mess. But, whoa, buddy.
I'm also a fan of websites publishing multiple lag numbers.
And lag stopwatching methodology published.
e.g. multiple numbers with corresponding lag-test disclosures.
"Lag of max Hz from Present() to GtG50%, screen center, VSYNC OFF" (more relevant to esports)
"Lag of max Hz from Present() to GtG50%, screen center, VSYNC ON" (more relevant to windows compositors & VSYNC ON gaming)
"Lag of 60Hz from Present() to GtG50%, screen center, VSYNC ON" (more relevant gaming consoles)
"Lag of max Hz from Present()+Flush() to GtG2%, screen top, VSYNC OFF, room temp calibrated 20C±0.5C, 24hour pre-warmup"
(more isolates display processing lag and warmup maximizes GtG speed, like a monitor left on 24/7)
Because of this, and if you start bypassing timestamping a GPU API and instead timestamp externally with a VSYNC detector on the video cable instead, I've seen <0.5ms lag and 20ms+ lag on the same monitor -- yes the spread is pretty huge.
Not possible to standardize around one number. The lag number on RTINGS is VSYNC OFF + GtG2%, so RTINGS will always be lower than websites that use VSYNC ON + GtG100%. Gigantic difference. Console users and PC-esports users need their own separate lag numbers because of obvious reasons clearly succulently mic-drop explained in this very post.
Timestamps of start is immediately after the Present() or Present()+Flush(), and never beforehand. GPUs have pipelining that does background rendering in a shingled manner (much like CPU instruction pipelining of running multiple instructions concurrently), so Flush() is important for removing GPU latency from lag test numbers by forcing the GPU to finish rendering the current frame and begin outputting the first raster scanline of that specific frame within microseconds of the flush, otherwise GPU lag can vary by multiple milliseconds (especially if GPU utilization is so low that it uses a low-power slower pipelining mode to save power).
Publishing latency test methodology is what we should shame "reviewer test methodology" (but not shaming reviewer names directly), because that's a big information deficit of input latency numbers. Blur Busters Advocacy on improving future lag testing will be optimized towards improved latency test disclosure. Sites won't unify testing methodology, but we can at least standardize a few important common use cases.
Broken record here. I've seen too many photodiode-tester users start reviewersplaining, and then become humbled once I fully explain how complex things actually is, with all the error margins they forgot about. Even for reviewers that aren't even influenced by manufacturers to choose a specific lag stopwatching parameters.
Yes, lots of messed up lag-testing methodologies. Being that said, a flawed methodology is still outputting numbers that are accurate to the flawed methodology and the methodology's own error margins. And sometimes "flawed methodology" is never unamiously agreed! Console users
should always view PC-esports lag numbers (VSYNC OFF lag tester) as flawed for them, and PC-esports users
should always view console-optimized lag numbers (VSYNC ON lag tester) as flawed for them. And that's not the only disagreement. Some of us wants to know the latency of the display itself, completely isolated from the GPU and GtG. There's so many needs that one number, just simply ain't possible.
Also, breaking in a newly-shipped LCD for 1 week of 24/7 operation is important for accurate GtG's too, because pressure spots (like the foam pieces of a monitor box, or mishipped flat causing a bezel-braket pressure imprint in the middle), take many days to fade and for GtG numbers to stabilize -- reviewers gotta break-in a newly received panel too. I've seen GtG numbers vary by more than 25%-100% in many squares between exactly the same panel for GtG heatmaps, just because a reviewer forgot to "break-in" a panel and also warm-it-up before testing too (e.g. some testers just power it on and begin testing right away -- but many like RTINGS will properly break in a monitor first, followed by making sure it's also pre-warmed up if it is already broken-in).
I also highly suggest you do not even bother to reply to this post, until you retest the darks of your GtG heatmap after putting your monitor temporarily in your basement box freezer for 30 minutes (simulate a cold LCD in a winter lab), and after 24 hours of re-warming back to room temperatures re-test after putting the LCD outdoors in direct summer sunlight (or hot car) for 30 minutes. You will be shocked at the major GtG differences and major lag differences. Totally. And many reviewers don't bother to publish their room temperature -- even 18C vs 22C has major lag differences if you're using a temperature-sensitive GtG trigger. Many pro testers will use a lab thermometer to calibrate their room temperature exactly to 20C. This was less important back in the "33ms" GtG days twenty years ago, but now critically important in the "1ms" GtG days.
TL;DR: The numbers are not lies, even if test methodology may be influenced, and sometimes diverge from real-world reaction time. They are just different methodologies, possibly executed in imperfect conditions (e.g. room temperatures, panel lottery effects, GPU pipelining performance).