[$2000 bounty] Calling 120Hz+ Linux Users: TestUFO Work/Fail

NEW for 2017: Discussion about the testufo.com Blur Busters Motion Tests. Widely used by enthusiasts, display tweakers, YouTubers reviewers, monitor manufacturers and VR headset makers!
User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by Chief Blur Buster » 05 Feb 2018, 12:38

RealNC wrote:Before opening the bug, this might be relevant prior knowledge:

https://bugs.kde.org/show_bug.cgi?id=322060#c117
Thank you for this pertinent information.
KDE Maintainer Martin Flöser wrote:* We lack developers with the expertise to understand the problem
* We lack developers with NVIDIA cards
* The last patch we did for NVIDIA specific issue caused severe issues which required an emergency release
* We have no chance to properly understand what's going on due to NVIDIA driver being proprietary
* If the NVIDIA driver as thee only driver needs such workarounds, NVIDIA should fix their drivers or contribute patches
If the problem is blocked by NVIDIA, the $1000 opensource bounty may not help.

But at least I have until mid-2019 to see if any movement can occur with some incentives.

e.g.
(1) an offer of a free NVIDIA GeForce series card (probably a midrange 900 series);
(2) an offer of a free "Better Than 60Hz" monitor to the same developer;
(3) an offer to teach developers how to understand motion fluidity. Blur Busters TestUFO and Blur Busters articles have educated many over the years.

To other visiting software developers: If any open source developer working on Chrome (chromium.org), KDE, and other pieces of puzzle, is interested, inquire within -- [email protected]

It can't be in code freeze all of 2018, after all.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

xorgy
Posts: 1
Joined: 23 Apr 2018, 19:47

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by xorgy » 23 Apr 2018, 20:01

Contrary to what some others in this thread have said, compositing is not required for vsync, nor is the TearFree mode. Chrome will V-Sync correctly if it is rendering the whole screen, or if TearFree is enabled (and functioning with your driver). Compositors render the whole screen, so if they are implemented correctly (and using the recent presentation APIs) they can also do tear-free vertical sync.

I have a "144"Hz (actually closer to, but not quite, 144000/1001Hz) display and a Radeon WX 7100 (Polaris, like RX 480, RX 580, etc.) running standard Mesa drivers, I have TearFree enabled, and V-Sync is accurate (although occasionally late, Chrome's fault really) in Chrome, it displays your tests just fine. I honestly don't understand this "VSYNC is not available on the Linux platform" message, because a) yes it is and b) it's exposed (made available) exactly the same way as it is on other platforms (requestAnimationFrame).

Not sure about the magicks behind the Animation Timing Deviation test which cause it to stutter unlike anything else, maybe it's a 2D canvas thing, it doesn't happen with WebGL applications or DOM in my experience. I'm inclined to think that the only thing causing this stutter is poor 2D canvas drawing performance, which I'm more sure of by increasing the number of pixels per plot (and thus the number of individual canvas commands).

Side note: One interesting thing is that the timing deviation plot looks a bit like a signal close to the nyquist limit, but not noise. It'd be interesting to take an FFT of this and see if there is some sort of predictable oscillation in timing deviation. In theory, the jitter should be random (since that is the goal of the recent security work to reduce the precision of performance.now), but it looks like it's oscillating at a glance.

I also took a look at a more precise tool at the address http://codeflow.org/issues/timing/ which demonstrates how stable normal applications tend to be in terms of frame timing in Chrome, here's what it showed me:
download.png
download.png (34.25 KiB) Viewed 17649 times
Attachments
How she looks to me.
How she looks to me.
atdplot.png (57.32 KiB) Viewed 17650 times

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by Chief Blur Buster » 24 Apr 2018, 17:39

Very good!
xorgy wrote:I honestly don't understand this "VSYNC is not available on the Linux platform" message, because a) yes it is and b) it's exposed (made available) exactly the same way as it is on other platforms (requestAnimationFrame).
TestUFO was launched in 2013. Back at the time, more than 90% of Linux users were unable to get smooth TestUFO motion, while more than 75% of Windows/Mac users could. A decision was made at the time to display that message. Things have improved today, but, it is still very patchwork.

The percentage is increasing, but analysis shows still less than 50% of >60Hz Linux users are able to successfully get skipfree TestUFO, while more than 90%-95% of Windows users now get skipfree TestUFO. Also, FireFox/Chrome can run TestUFO at 480 frames per second under Windows now, the newer version of Chrome now fixed a 480Hz issue and also flawlessly works -- see 480Hz Monitor Tests -- we were the first in the world to test 480Hz.

The VSYNC accuracy of an average random end user's system is greatly improving. For Android platforms, it has improved to the point where I deleted the "VSYNC is not available on Android" message, but has not yet improved to the point where I'm ready to delete the "VSYNC is not available on Linux" generic message, for a random distribution.

Yes, these should link to FAQs instead of a short message (Eventually...). Right now, many motion tests run under the implicit assumption that VSYNC is perfect, such as our scientific pursuit camera tests and we are unable to detect if VSYNC fails on Linux because sometimes it looks like a perfect 60Hz display when it's actually a 60Hz timer.

Web browsers don't give system information about whether VSYNC is truly supported or not, so we have to rely on heuristics (the noisepatterns of animation time graph) and/or common numbers ("144") to assume that it's not running a hardcoded 60fps timer that's completely unsynchronized to VSYNC. Since that can't be scientifically as guaranteed as on a Windows/Mac -- the heuristics are so good on Windows platforms that I am reliably detecting single-framedrops right on cue in Google Chrome. This made it possible to do vision science with web browsers since on some platforms, seeing green "VALID" means the animation is (almost virtually) guaranteed precise because I'm realtime detecting frameskips via heuristic algorithms.

But I cannot do that with >75% of Linux setups, and I can't inform users that their web browsers are running flawlessly stutterlessly.

Believe me, as a gold standard that multiple gaming monitor display reviewers use (TomsHardware, TFTCentral, RTings, TechPowerUp, PCMonitors, HDTVTest, etc), the green READY needs to be virtually practically guaranteed to be stutter free (as much as javascript heuristics feasibly allows me), as we are also commonly used for monitor overclocking (TestUFO Frame Skipping needs to be guaranteed running flawlessly without browser frame skipping, for reliable diagnosis of monitor overclocking -- all frameskips must be caused by the monitor, not by the web browser.

Google "Monitor overclocking frameskipping test" and you'll see lots of forums/sites/youtubers/etc link to us as the world's gold standard frame skipping test for computer monitor overclocking. Anybody who's using a frameskipping test, are using our frameskipping test. We're the gold standard, because we're the only one that successfully reject stutters by displaying a results-invalidating popup. Because we successfully know if it's a browser stutter or a monitor stutter.

TestUFO tests require a 90%-reliable framedrop-detection guarantee, so I can notify users if there are stutters. That allow users to re-run scientific tests if a single stutter is detected. TestUFO is used for vision research and display testing science, etc. I currently can't get that 90% with Linux (yet).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by Chief Blur Buster » 24 Apr 2018, 17:49

xorgy wrote:Not sure about the magicks behind the Animation Timing Deviation test which cause it to stutter unlike anything else, maybe it's a 2D canvas thing, it doesn't happen with WebGL applications or DOM in my experience. I'm inclined to think that the only thing causing this stutter is poor 2D canvas drawing performance, which I'm more sure of by increasing the number of pixels per plot (and thus the number of individual canvas commands).
Yes, it's the number of individual commands for the graph. The graph uses more draw calls than most TestUFO tests. It runs smoothly at 120fps on my 4K120Hz display on Windows however.

And it's possible your Linux browser may be less forgiving of frame rendertimes:

Image

These are my tests from August 6, 2013. See my old article. You will observe that Chrome stays perfectly smooth even if drawcalls takes 10ms out of 16ms. To do these tests yourself, add a "&busywait=<milliseconds>" to the TestUFO animation-time-graph URL.
  1. 1 millisecond busywait
  2. 2 millisecond busywait
  3. 3 millisecond busywait
  4. 4 millisecond busywait
  5. 5 millisecond busywait
  6. 6 millisecond busywait
  7. 7 millisecond busywait
  8. 8 millisecond busywait
  9. 9 millisecond busywait
  10. 10 millisecond busywait
  11. 11 millisecond busywait
  12. 12 millisecond busywait
  13. 13 millisecond busywait
  14. 14 millisecond busywait
  15. 15 millisecond busywait
  16. 16 millisecond busywait
Click on these. One by one. For your Linux browser.
Watch for stutters during graph scrolling.
I can go to 5 or 6 before it stutters at 144Hz.
I can go to 14 or 15 before it stutters at 60Hz.

I can go almost all the way to a refresh cycle length's worth of busywaiting in Windows Chrome, before a SINGLE stutter appears. But some platforms, and some browsers just give up the ghost after 1 or 2 or 3ms of busywait. That makes them particularly sensitive to rendertime fluctuations, and sometimes it's erratic. In Chrome on Windows, with GPU enabled, the browser is darn near practically guaranteed not to stutter provided the frametime is less than a third of a refresh cycle and is currently in sync with the previous rendertime.

Often, many Linux systems, drivers, window managers, provide an unpredictable environment to guarantee vision science with the browser. Your system is very, very, very good. But what about an average, random system? Even a nearly netbook-era computer now runs 60fps TestUFO fine in WIndows, at least for the simpler tests on TestUFO, thanks to the generous rendertime jitter margin of Chrome. Can we improve Linux all-window-managers-wide, and all-drivers-wide, Intel, AMD, NVIDIA, all of them, to match that? Please?

Blur Busters is offering a $1000 open source bounty for reliable Linux VSYNC in default browser installs on 5 distributions

Option A
Open source bounties offered (up to USD $1000.00 total -- personal funds out of my pocket -- offer valid through June 31, 2019). The requirements are: Stutterfree in same browser for all configs to 50% refresh cycle rendertimes (busywait is an acceptable simulation of long rendertimes) for 60Hz, 120Hz and 144Hz (and no framecaps, so 240Hz/480Hz allowed but not required as part of bounty), all CPUs/GPUs made in last 5 years, must include Intel/AMD/NVIDIA (all open source and closed source drivers), default Linux distribution install configs of all 5 most popular Linux distributions. Random-chance tests being 90% successful on a sampling of 10 random Linux systems (9 out of 10 randomly chosen Linux combos meeting above criteria 100% skipfree in 15+ second sustains at 8ms rAF() frametimes for 60Hz, 4ms rAF() frametimes at 120Hz, and 3.5ms rAF() frametimes at 144Hz -- and if a rare frameskip occurs on a lightly loaded system, then those single-frame frameskips have to be successfully detectable more than 90% of the time. (occasional False positives OK like 10%-of-time accidentally thinking there was a stutter when there was none -- but there should almost never be false negatives; standard is set at 1% -- only 1% of stutters should go undetected. This threshold is negotiable, however). When claiming bounty, BlurBusters reserves discretion in choosing random Linux systems to test on before awarding bounties (in countries where it is legal) and adjusting bounty for any shortfalls in meeting goals. Qualifying browsers include Chrome or FireFox (can focus on one or the other, or both). Some won't need modification but some will. Currently Windows Chrome is a PASS under this criteria as I can get the TestUFO green-READY guaranteeing frameskipfree operation >90% of the time on a random native-boot Windows system now that meets the "5-years-old-and-newer" criteria on its CPU and GPU. (except for virtual machines -- that's a mess -- don't worry about virtual machines). I can expand this into a more formal list of requirements upon request for interested programmers. We have to clean up the stutterfest mess as much as possible. It's getting better, but not getting better fast enough -- it has to be 90% successful by random chance out-of-box with no further user configuring. I can grab a random Windows system, run TestUFO, and reliably self-detect framedrops 90% of the time in the Chrome browser specifically (which is why I recommend Windows users run Chrome with TestUFO), but I'm impartial to any browser willing to provide reliable data I can act upon. Chrome has always acted fast on 120Hz BugZilla issues so mucho kudos for that specific part. Also, 5-years-and-newer is a generous allowance, because TestUFO runs smooth at 60fps on my decade-old netbook, so I have offered more lenient rules and a lower bar, than the current Windows 7-8-10 situation.

Option B
Help convince all browser vendors or W3C to definitively properly detect missed VSYNC blanking intervals (via a driver standardized way) in a properly configured web browser. Bonus if it also includes something similar to RasterStatus.INVBlank but in a cross platform way. Basically if I can detect existence of native VSYNC support, and I can reliably self-detect framedrops (to notify users about them), and I'm given a frameskip-free guarantee through up to at least 50% refresh cycle busywait. Then I don't have to use useragent discovery to decide if the browser is trusted for frameskip-detection-via-heuristics. I have one commit to my claim, Section 7.1.4.2 of the HTML 5.2 standard -- collaborating with the W3C in this topic area.

TL;DR: While I do plan to eventually reword the message -- It is an absolutely critical requirement that >90% confidence is made possible before I display a green "READY". Less than 10% of Windows systems have issues like "TestUFO Stuck at 60Hz" and such. I need to get scientific reliability in stutter-detection capability like I can with Chrome under Windows. Hundreds of hours of stutter analysis has been done over the years with browser capability to run perfect TestUFO animations.

Note: Special negotation is available to extend the deadline beyond mid-2019, since I realize it may take a year or more before distributions adopts standards. Random graphics cards, random graphics drivers, random computers containing one of the top5 Linux distributions (top5 will be queried at the end of bounty) -- I realize it takes a long time for fixes to filter through the whole chain (aka good vaccine) before the whole system is sufficiently reliable that a random system is likely to work. Extra funding may be available on special negotiation.

I'm all open to ideas!
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

jigoku
Posts: 8
Joined: 27 May 2018, 12:47

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by jigoku » 30 May 2018, 13:59

Hi, just solved a framerate issue i was having in firefox, using a workaround for "layout.frame_rate", setting this to the actual refresh rate allows you to get accurate comparisons of the UFO tests, that can be compared to visual results on a Windows machine. Since setting "layout.frame_rate" to 120, my lightboost monitor is now able to render the ghosting tests and others without artificial motion blur.

Thought i'd post this here in case it helps anyone else :
viewtopic.php?f=4&t=4170#p33386

(edit; only just realised the link was broken, fixed)
Last edited by jigoku on 03 Jun 2018, 14:36, edited 1 time in total.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by Chief Blur Buster » 31 May 2018, 10:06

jigoku wrote:Hi, just solved a framerate issue i was having in firefox, using a workaround for "layout.frame_rate", setting this to the actual refresh rate allows you to get accurate comparisons of the UFO tests, that can be compared to visual results on a Windows machine. Since setting "layout.frame_rate" to 120, my lightboost monitor is now able to render the ghosting tests and others without artificial motion blur.

Thought i'd post this here in case it helps anyone else :
posting.php?mode=quote&f=4&p=33386
Thanks for the follow up -- indeed hardcoded framerates can help although there can be a small bit of slew between framerate (e.g. 119.995fps) and refreshrate (e.g. 120.017Hz) causing the occasional frameskip.

This is because the GPU clock is not in perfect sync with the CPU clock, so there is always a real-world difference even if config is exactly matched.

Even thermal changes and other factors (power management) can cause slew differences (in the microseconds league).

So a hardcoded number isn't the end-all-to-end-all ultimate solution - just a workaround that can make some users happy for now.

Also known side effect: If the framepacing is not made very impeccable (microsecond accurate), a jittery frametime may jitter across a VSYNC threshold causing random surges of stuttering at a beat-frequency threshold between fps and Hz (smooth-stutters-smooth-stutters-smooth-etc effect). In the best case, there is one frame skip + a sawtoothing input lag. If framepacing is accurate enough, it is just a single stutter, and is not noticed by end users if it only occurs once every ten seconds and overwhelmed by web browser random stutters.

In the long term, the framerate needs to achieve perfect refresh rate sync out of the box.

I can run TestUFO on Windows and it runs perfect zero frameskip on 90% of systems manufactured in last 5 years (including plain Intel GPUs) for a solid ten minutes at 60Hz if nothing running in background. And if a stutter does indeed occurs, it is almost always successfully detectable. I've relaxed the requirements of my $1000 bounty for making common Linux installs acheive scientifically-accurate TestUFO out-of-box 90%-of-time using default settings.

At least determined users can do a system-specific solution - so thank you for posting about this. Appreciated!
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
RealNC
Site Admin
Posts: 3737
Joined: 24 Dec 2013, 18:32
Contact:

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by RealNC » 21 May 2019, 23:47

There has been a recent development. A fork of KWin has appeared that lets vsync drive the compositing loop.

https://github.com/tildearrow/kwin-lowlatency

This completely fixes all input lag and stutter issues in KDE for me.

Image
SteamGitHubStack Overflow
The views and opinions expressed in my posts are my own and do not necessarily reflect the official policy or position of Blur Busters.

yasamoka
Posts: 1
Joined: 21 Aug 2019, 22:08

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by yasamoka » 21 Aug 2019, 22:10

RealNC wrote:There has been a recent development. A fork of KWin has appeared that lets vsync drive the compositing loop.

https://github.com/tildearrow/kwin-lowlatency

This completely fixes all input lag and stutter issues in KDE for me.

Image
You're a saint! Thank you very much for pointing us to this. I just compiled and installed it and FINALLY, my issues with Firefox stuttering like mad @ 165Hz have, for the most part (99%), vanished.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by Chief Blur Buster » 22 Aug 2019, 15:58

That's fantastic. Very glassfloor too, it looks like it could easily do 480 Hz+.

This isn't default in any single distribituion though, lest 5 -- required for the Blur Busters $1000 bounty -- so still a long way to standardize proper high-Hz VSYNC on Linux.

This needs to become standard in all distros, stat.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

life_at_1ms
Posts: 38
Joined: 31 Oct 2019, 03:20

Re: Calling 120Hz+ Linux Users: TestUFO Works/Fail?

Post by life_at_1ms » 24 Nov 2019, 06:47

Glad I found this thread... Is there a simple and reliable way of getting 144hz+ across a Linux desktop experience? So you can run Looking Glass on Linux and get 240hz in a Windows guest, but still only 60hz on the Linux host? This hurts... I want 144hz+ in my browser, terminal, and the rest of my Linux desktop windows. I'm happy to use Fluxbox, Openbox, or a similarly lightweight WM (I'm guessing the kwin workaround is KDE only). I'm also happy to purchase special hardware to guarantee it.

Thanks for the research everyone!

Post Reply