So what refresh rate do I need? [Analysis] [very good one!]

Advanced display talk, display hackers, advanced game programmers, scientists, display researchers, display manufacturers, vision researchers & Advanced Display Articles on Blur Busters. The masters on Blur Busters.
ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 24 Feb 2014, 05:22

Chief Blur Buster wrote: Conclusion/TLDR:
- Scanning creates tilt effect on motion perpendicular to scan direction;
- CRTs and non-strobed LCD will have the scan tilting effect;
- Higher refresh rates / faster scanning will have less tilting effect;
- LightBoost (and similar strobe backlights) will not have the scan tilting effect;
- Multiscanning will create stationary tear-line artifacts;
- Interlacing will create venetian blind artifacts;
- Anything other than all-at-once presentation can cause motion distortions, in one form or another;
- Sequential scanning is the least-distorting method of scanning, short of doing all-at-once presentation.
What you forgot to mention is input lag variation - input lag at the top of the screen will be lower then at the bottom.
Of course you could race the beam (rendering each scanline seperately) or use continuous predictive time warp with eye-tracking, but these come with big drawbacks.

As input lag variation (not total lag) and visual instability cause nausea among other issues, it is preferable to do away with raster-scan altogether and use simultaneous refresh- what Valve called 'global display', using their custom made display drivers for their latest VR prototype.
Side note: To do this, you need to need to pre-buffer the image, but at maximum scanout speed with variable v-blank (think g-sync), lateny can be reduced to non-problematic levels.
Our prototype uses global display, where all pixelsare illuminated simultaneously. This avoids the
compression, stretching, and tilting problems that can occur with the more standard rolling display,
where pixels are illuminated in a scanned sequence over the course of a frame. It may be that the
artifacts of rolling display can be largely corrected by adjusting the frame buffer during each frame to
account for eye motion, but that’s not yet proven; also, while head motion can often be used as a
proxy for eye motion, without eye tracking there will always be failure cases, and low-latency headmounted eye tracking is not a solved problem. So right now global display is the only approach known
to work.
Further reading:
http://blogs.valvesoftware.com/abrash/l ... ar-and-vr/
http://blogs.valvesoftware.com/abrash/r ... s-the-eye/
http://www.altdevblogaday.com/2013/02/2 ... trategies/
http://media.steampowered.com/apps/abra ... 202014.pdf

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 24 Feb 2014, 10:32

ScepticMatt wrote:What you forgot to mention is input lag variation - input lag at the top of the screen will be lower then at the bottom.
Oh, I'm already familiar with that;

However, I have considered that to be information to be a new subtopic of this topic -- worthy of another huge bunch of diagrams. The Leo Bodnar input lag tester, for example, has multiple flashing squares, for top, center, and bottom, since they are different measurements relative to the beginning of the scan. LightBoost is known to have a bit more input lag precisely because of this; the top edge of screen has a full frame more lag, the center of screen has half a frame of lag, and the bottom has virtually no lag delta. This is because of the delayed presentation of the top/center since it's scanned in the dark and then the whole screen is flashed at once after the scan hits the bottom and the LCD pixels have essentially finished transitioning (GtG).

There are also hard-to-understand input lag interactions with VSYNC OFF for framebuffered architectures, that required diagrammed explaining, like simplified illustration of input lag of different parts of the screen during VSYNC OFF at 240fps @ 120Hz, which I'll quote here for completeness' sake:
Chief Blur Buster wrote:
Q83Ia7ta wrote:What about center and bottom of the screen? If understand right with vsync off bottom of the screen will have more "input lag" than the top?
Explaining input lag of different parts of the screen during VSYNC OFF is kind of complicated, due to a lot of variables.

Generally speaking,
- Whether VSYNC ON, VSYNC OFF, or GSYNC -- all monitors are still doing top-to-bottom scanning (as seen in high speed video)
- During VSYNC ON and VSYNC OFF, the timing of the scans are synchronous (e.g. exactly 60 times a second, 1/60sec apart)
- During GSYNC, the timing of the scans are asynchronous (e.g. occurs when the GPU finishes rendering a frame)
- During all modes, the amount of time a scan takes place is constant (e.g. scans occurs from top-to-bottom at constant velocity)
- During VSYNC OFF, the input lag above the tearline is always higher than the input lag below the tearline. This skews the timings a bit.
- During strobing, you've got all-at-once presentation (regardless of the monitor mode, VSYNC ON or VSYNC OFF) because the monitor was scanned in the dark but then flashed all at once. This can skew the time-basis of the presentation timing of the slices of refreshes during VSYNC OFF. (This can affect responsiveness)

During a theoretical infinite framerate VSYNC OFF on an instant-response monitor (e.g. CRT), you're always real-time at the position of the scan line, you've eliminated tearing, and you've got minimum possible input lag by having the freshest possible input. You still only get a discrete number of samples per second for each pixel, but whenever that pixel is displayed, its input lag would be very fresh.

Now, if you're running at 240fps at 120Hz, let's say you've got a tearline in exactly the middle of the screen. Let's assume the game does input reads everytime it renders a new frame (not all do, but this creates the best-case scenario for any given frame rate). So you've got half of a previous frame displayed in the top half of your monitor refresh and half of the subsequent frame displayed in the bottom half of your monitor refresh. For a real-time zero-buffered display, you've got the same amount of average input lag for the top half, and for the bottom half, because the top half was displayed (scanned out) slightly earlier to your eyes, and the bottom half was displayed (scanned out) slightly later to your eyes. So this results in the top half and bottom half having the same input lag. But let's measure WITHIN the slices. Within the top half, you've got less input lag for the bottom part of the top half (the image right above the tearline). And within the bottom half, you've got less input lag for the bottom part of the bottom half (the image right above the bottom edge of the screen). Confused yet? It gets more complicated. Tearlines varies all the time in positions, you might have two in a refresh. It's even harder to explain/measure input latency in these situations.

Explaining all of this very well, requires a very well-written article, which Blur Busters plans to do after all the important monitor reviews are done. But I will be happy to dive into small morsels at a time (even these takes a lot of explaining).

A picture is worth a thousand words, so here is one I've created just now:

Image

In real life, the tearlines will be all over the place, as 240fps often won't be perfectly synchronized with 120fps. However, this simplified diagram gives you a taste of the complexity of input lag during VSYNC OFF situations. There are other variables left out, such as excluding cable lag and pixel transition lag, but this gives the correct general idea, and also additionally, helps explains why input lag continues to go down even as framerates go higher and higher above your refresh rate.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 24 Feb 2014, 12:09

Hmm, I didn't realize that light-boost display scann out 'in the dark'. Very nice.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 24 Feb 2014, 12:27

All strobe backlight monitors essentially behave as global displays (LightBoost, ULMB, Turbo240, BENQ Blur Reduction) as they scan the LCD panel in the dark (unseen by eyes), and then flash the backlight all at once at the end. The transitions (GtG) is unseen by eyes, and motion blur now controlled by strobe length, and latency is consistent relative to GPU frame buffers (during VSYNC ON operation)

Global displays (Displays that present all-at-once) always has more input lag than scanned displays, because of the finite video signal transmission speed from the computer to the display. All-at-once presentation requires buffering (or scanning in dark) before visual presentation. While sequential scanning displays can take advantage of this by real-time scanning-out. However, for framebuffered architectures and VSYNC ON operation, global displays have the least input lag variations, for situations where latency variations are a problem, and you need the most perfect motion possible (framerate == refreshrate synchronized motion without tearing, e.g. capped-out VSYNC ON).

Also, some of these displays (e.g. LightBoost) actually does some partial buffering and then does an accelerated scan-out. This is in order to create a longer pause between refresh cycles, to allow LCD pixel transitions (GtG) to be more complete, before flashing the strobe backlight. (See the "How it Works" section at http://display-corner.epfl.ch/index.php/LightBoost ...). So LightBoost effectively, behaves as a global display, to the human eye.

Scanning may still be needed for some architectures such as OLED, since they are scanned displays, and rolling-scans is a low-latency method of low persistence from OLED. Fast scanning (e.g. refreshing a display in 1/1000sec) can also make skew/compress effects negligible, as near-instantaneous scanning start to resemble a global display more and more.

Did You Know? VSYNC ON wasn't always evil. Old games such as Super Mario Brothers and Street Fighter were always VSYNC ON. The classic 8-bit and 16-bit era nearly always operated synchronized to refresh rate (VSYNC ON), all the way back to the Atari 2600 systems which was by its nature, graphics always synchronized to refresh since everything had to be done in realtime as there was almost no memory. We never complained about input lag for those games. But as latency increased with 3D rendering and framebuffered architectures, VSYNC ON became evil in input lag. However, as we hit 120fps and beyond (240fps @ 240Hz on future displays), VSYNC ON latency becomes less and less, and should hopefully become acceptable again.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

dreamss
Posts: 6
Joined: 05 Jan 2014, 08:02

Re: So what refresh rate do I need? [Analysis] [very good on

Post by dreamss » 25 Feb 2014, 04:10

ScepticMatt wrote:
Chief Blur Buster wrote:Related Topic: Long-time competitive FPS players are often so tuned to a specific latency of a specific game/system configuration, that they will notice small changes to latency they're pre-trained to. (One easier way to picture how much this matters is, some of them aim a mouse at a fast speed of several screenwidths per second panning. Say, the equivalent of 5000 pixels/second mouse movement speed to move something into crosshairs. They decelerate to put crosshairs on objects. But add a 10ms lag creates a 50 pixel overshoot, which they then have to correct for. Just 5ms lag creates a 25 pixel overshoot. Competitive players can feel they're taking longer to aim their crosshairs on a target if the latency is different from what they've been pre-trained to. They aim, correct maybe once or twice, and shoot. For example, aim, overshoot, move mouse back 20 pixels, aim perfect. But change latency and it becomes aim, overshoot, move mouse back 50 pixels, overshoot again, move mouse back 20 pixels, overshoot again, move mouse back 5 pixels, aim perfect. That back-and-forth swerving of mouse slows down aiming since the competitive FPS player mistimed the moment to begin decelerating the mouse. Just a few milliseconds change in whole-chain latency causes this to happen, and they 'feel' they're taking much longer to aim, and they complain. I'm not sure if there's any science papers done specificially on professional FPS players -- perhaps you'd be able to call up some?
I don't know about competitive gamers, but there are performance tests of input devices standardized in ISO 9241-400 (formerly ISO 9241-9)
examples: http://www.cse.yorku.ca/~rteather/pdfs/3dui09.pdf
http://www.cas.mcmaster.ca/~teather/pdfs/3dui11.pdf
i can attest to this, its purely muscle memory. for example the drag from a dirty or sweaty mousepad will trow my aim off, if its 100% smooth i can easly snap without even thinking about it. atm im using a teflon based mousepad and added a custom ptfe skate to the mouse. i find the default mini skates mouses come with gather sticky dirt which trows me of by a lot

this looks rather getho, but i be damned if the drag is smooth and even all across the mousepad

Image

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 27 Feb 2014, 10:48

A good post from a separate thread, but since I've tweeted/blogged about this thread, I'm including a copy here for completeness' sake:
ScepticMatt wrote:What is temporal aliasing and why is it a problem?

Due to the time-discrete nature of our displays, transformation speeds (movement, rotation) less then half the frame rate causes false detail. For example, a wheel rotating at 120 Hz filmed by a 24Hz camera with very low exposure duration speeds appear static, when they should be blurred ('wagon-wheel effect). Without motion blur, computer generated images show aliasing even at 10,000 frames per second. Therefore, computer graphics have to simulate motion blur to reduce it.

Rendering Examples:
Assume we have a 100 Hz display with 2ms persistence and 100% fill factor.
1000x1000 Pixels, 50 degree FoV.

Example 1. Stroboscopic effect:
Movements faster than 1 pixels per frame causes objects to jump.
Setup: Fixed eye, pixel moves at 233 pixels per second from left to right

No aliasing - inconsistent, jumpy movement (yellow=ideal)
Image
Spatial aliasing makes the movement time-consistent, but gaps remain
Image
Directional motion blur reduces gaps, but inconsistency remains
Image
A combined approach is needed.
Image

Example 2: Eye tracking:
Eye tracking needs to be accounted for, especially when the screen has a large FoV, such as head mounted displays. Same setup as Example 1 with motion blur, but now our eyes track the movement of the pixel. Pixel movement is blurred, but should be sharp. Display-static objects have an apparent motion of 233 pixels per second from right to left and suffer from stroboscopic effects.
Image

Example 3. Phantom array effect:
Lower persistence reduces blur during saccades, disabling saccadic surpression and causing perceived backward motion.
Setup: 20 degree horizontal saccade (25ms duration), static pixel (pixel width = 0.05 degree)
blue line = saccadic movement
black line = hold-type blur (non-linear as the eye-velocity changes)

2ms persistence. Up to 66 pixels blur (3.3 degree)
Image
Full persistence. Up to 177 pixels blur (8.8 degree)
Image

Conclusion:
Time discrete rendering causes temporal aliasing. As resolutions increase, impractical frame rates are necessary to avoid it. Spatio-temporal anti-aliasing can converge to an ideal solution, thus eliminating the need for higher frame rates (beyond flicker fusion), but eye movements need to be accounted for.

Limitations:
In practice it isn't possible to deal with temporal and spatial aliasing separately and achieve 'perfect' motion blur. See:
Shinya et al. 1993 / Spatial Anti-aliasing for Animation Sequences with Spatio-temporal Filtering
Fast eye-tracking system is still an unsolved problem for head-mounted displays.
Furthermore, as seen in example 3, the complex relationship between the human visual system and object motion is still an active area of research, and can improve effectiveness and efficiency of motion blur rendering techniques.

Annex: Overview of algorithms.
Image
Navarro et. Al 2010 / Motion Blur Rendering: State of the Art
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 27 Feb 2014, 11:37

Can you edit the above when I've fixed the images?

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 27 Feb 2014, 11:40

ScepticMatt wrote:Can you edit the above when I've fixed the images?
Yes, I can -- (or if you repost so it's under your name for future editing, and then I can delete my copy)
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 27 Feb 2014, 14:20

Should be fixed now - it ended up being more complicated because eye velocity during saccades isn't constant.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 28 Feb 2014, 10:15

Edited the message, it is now updated.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

Post Reply