So what refresh rate do I need? [Analysis] [very good one!]

Advanced display talk, display hackers, advanced game programmers, scientists, display researchers, display manufacturers, vision researchers & Advanced Display Articles on Blur Busters. The masters on Blur Busters.
ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 21 Feb 2014, 17:37

Chief Blur Buster wrote:So I'm really interested in seeing about new science papers about this new era when LCDs finally began to hit near theoretical maximum motion clarity efficiencies (GtG no longer meaninful part of motion blur calculations; and motion blur completely controlled by strobe lengths and unaffected by GtG/diffusion/other inefficiencies).
Sorry, I found nothing newer yet.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 21 Feb 2014, 17:43

Hopefully, researchers are working on these newer papers, or if you know some -- maybe you can influence them to study these areas?

With motion clarity on some LCDs now controlled by strobe length, there's no stopping LCDs from achieving 0.1ms (or less) persistence, especially with RGB LEDs which do not use phosphor. The limiting factor would simply be backlight brightness to compensate for the short strobe lengths.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 21 Feb 2014, 18:41

Have you read about temporal upsampling/time warping and motion compensated inverse filtering yet? What do you think about that?

Edit: Looking at http://www.testufo.com/#test=framerates-text
It demonstrates the issues with brightness dependence very well. At minimum screen brightness in a bright room far enough away, I can't detect the difference between 30 and 60 Hz anymore (minus extra blur, of course)
As I increase the screen brightness, the effects of low frame rate become more and more obvious.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 22 Feb 2014, 15:59

ScepticMatt wrote:Have you read about temporal upsampling/time warping and motion compensated inverse filtering yet? What do you think about that?

Edit: Looking at http://www.testufo.com/#test=framerates-text
It demonstrates the issues with brightness dependence very well. At minimum screen brightness in a bright room far enough away, I can't detect the difference between 30 and 60 Hz anymore (minus extra blur, of course)
As I increase the screen brightness, the effects of low frame rate become more and more obvious.
Not just brightness dependance, but also other variables:
-- angular tracking speed. The slower the tracking speed, the harder. The further you view, the slower the angular tracking speed.
-- size of object covering your vision.

Zoom the browser. Use 200% zoom while viewing http://www.testufo.com, then view from twice the distance. The relative angular movement speed becomes the same, and the apparent brightness (volume of photons) hitting your eyes become the same again. Thus, the difference between 30Hz vs 60Hz becomes equally as obvious again, except you now only have half the FOV to track the moving object with because the screen is covering half the width in terms of field-of-vision. At some point, you need a bigger display to allow enough time to track the moving object across your FOV.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 22 Feb 2014, 16:03

ScepticMatt wrote:Have you read about temporal upsampling/time warping and motion compensated inverse filtering yet? What do you think about that?
Yes, I have. Some of those ideas (or similar) are now being used in good interpolation found in HDTV's nowadays.

The problem is a lot of good interpolation require lookahead, which requires buffering, which creates input lag. That's no good for computers and gaming, so interpolation is DOA for computer use, until it is sufficiently low latency (e.g. 240fps @ 240Hz, 1 frame lookahead falls to only 4ms), then it could be a possible conduit to getting flicker-free low-persistence (1000fps@1000Hz without strobing) and without needing more than 240fps native framerate.

Lookahead is critical for good interpolation, to factor in motion acceleration and sudden changes of motion direction, etc. And you've got the occulsion detection problem. Imagine a FPS game with an exploding object behind a chain link fence -- spinning debris in all directions at different speeds going through occulsion effects. That will never be perfectly interpolated, and visual artifacts are always a problem.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 23 Feb 2014, 05:08

Chief Blur Buster wrote:
ScepticMatt wrote:Have you read about temporal upsampling/time warping and motion compensated inverse filtering yet? What do you think about that?
The problem is a lot of good interpolation require lookahead, which requires buffering, which creates input lag. That's no good for computers and gaming, so interpolation is DOA for computer use, until it is sufficiently low latency (e.g. 240fps @ 240Hz, 1 frame lookahead falls to only 4ms), then it could be a possible conduit to getting flicker-free low-persistence (1000fps@1000Hz without strobing) and without needing more than 240fps native framerate.

Lookahead is critical for good interpolation, to factor in motion acceleration and sudden changes of motion direction, etc. And you've got the occulsion detection problem. Imagine a FPS game with an exploding object behind a chain link fence -- spinning debris in all directions at different speeds going through occulsion effects. That will never be perfectly interpolated, and visual artifacts are always a problem.
Lookahead improves filtering, but isn't necessary. Blurred frame insertion with mesh based warping works well enough without it, and lag lies in the few ms rage at most. Disoccoclusion is the only issue remaining, but problems can be reduced a lot in stereo rendering, and selective weighting can remove artifacts at the cost of effectiveness of the strobing reduction.
See http://www.mpi-inf.mpg.de/resources/Ada ... thesis.pdf
And http://www.iryoku.com/smaa/downloads/SM ... iasing.pdf
'our temporal': using stereoscopy and past frames
Image

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 23 Feb 2014, 11:45

That one is a very good one that works great for video material.
And possibly even games too -- if the latency numbers can be brought down to single-digit milliseconds.

I can definitely see interpolation (without lookahead) working with games/etc is if it's game-engine-assisted or GPU-assisted, where the GPU/game gives some general additional supplemental information to the interpolation logic (e.g. current & predicted motion vectors, or even predictability information: e.g. motion vectors suddenly change or there's a scene change) so that the accuracy of interpolation without lookahead, becomes artifact-free. There would be a question of whether it's cheaper silicon-wise to do interpolation from 120fps->1000fps, than to simply do direct GPU rendering.

If we still need lookahead to maintain quality, I think 1-frame lookahead becomes practical in input lag perspective (~4ms) only when we hit about 240fps. Possibly just 120fps (~8ms lag) is tolerable to all but elite competitive gamers, if processing overhead is reduced to almost zero (e.g. interpolated frames are generated virtually instantly).

We have these additional stringent variables to worry about:
- Interactivity where humans can get bothered by small latency deltas between action and reaction. This is a massive problem for VR and for touchscreens (Microsoft Research - YouTube). This is a smaller problem for mouse-screen coordination, so interpolation may be more workable for monitors than for VR displays...
- Ultra sharp graphics of computer graphics. People notice the difference in AA being turned on/off -- that's single pixel changes. Interpolation will cause distortions that are far more noticeable in games than in video.

phpBB [video]


Related Topic: Long-time competitive FPS players are often so tuned to a specific latency of a specific game/system configuration, that they will notice small changes to latency they're pre-trained to. (One easier way to picture how much this matters is, some of them aim a mouse at a fast speed of several screenwidths per second panning. Say, the equivalent of 5000 pixels/second mouse movement speed to move something into crosshairs. They decelerate to put crosshairs on objects. But add a 10ms lag creates a 50 pixel overshoot, which they then have to correct for. Just 5ms lag creates a 25 pixel overshoot. Competitive players can feel they're taking longer to aim their crosshairs on a target if the latency is different from what they've been pre-trained to. They aim, correct maybe once or twice, and shoot. For example, aim, overshoot, move mouse back 20 pixels, aim perfect. But change latency and it becomes aim, overshoot, move mouse back 50 pixels, overshoot again, move mouse back 20 pixels, overshoot again, move mouse back 5 pixels, aim perfect. That back-and-forth swerving of mouse slows down aiming since the competitive FPS player mistimed the moment to begin decelerating the mouse. Just a few milliseconds change in whole-chain latency causes this to happen, and they 'feel' they're taking much longer to aim, and they complain. I'm not sure if there's any science papers done specificially on professional FPS players -- perhaps you'd be able to call up some?
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

ScepticMatt
Posts: 36
Joined: 16 Feb 2014, 14:42

Re: So what refresh rate do I need? [Analysis] [very good on

Post by ScepticMatt » 23 Feb 2014, 16:06

Chief Blur Buster wrote:Related Topic: Long-time competitive FPS players are often so tuned to a specific latency of a specific game/system configuration, that they will notice small changes to latency they're pre-trained to. (One easier way to picture how much this matters is, some of them aim a mouse at a fast speed of several screenwidths per second panning. Say, the equivalent of 5000 pixels/second mouse movement speed to move something into crosshairs. They decelerate to put crosshairs on objects. But add a 10ms lag creates a 50 pixel overshoot, which they then have to correct for. Just 5ms lag creates a 25 pixel overshoot. Competitive players can feel they're taking longer to aim their crosshairs on a target if the latency is different from what they've been pre-trained to. They aim, correct maybe once or twice, and shoot. For example, aim, overshoot, move mouse back 20 pixels, aim perfect. But change latency and it becomes aim, overshoot, move mouse back 50 pixels, overshoot again, move mouse back 20 pixels, overshoot again, move mouse back 5 pixels, aim perfect. That back-and-forth swerving of mouse slows down aiming since the competitive FPS player mistimed the moment to begin decelerating the mouse. Just a few milliseconds change in whole-chain latency causes this to happen, and they 'feel' they're taking much longer to aim, and they complain. I'm not sure if there's any science papers done specificially on professional FPS players -- perhaps you'd be able to call up some?
I don't know about competitive gamers, but there are performance tests of input devices standardized in ISO 9241-400 (formerly ISO 9241-9)
examples: http://www.cse.yorku.ca/~rteather/pdfs/3dui09.pdf
http://www.cas.mcmaster.ca/~teather/pdfs/3dui11.pdf

hwps
Posts: 1
Joined: 23 Feb 2014, 19:03

Re: So what refresh rate do I need? [Analysis] [very good on

Post by hwps » 23 Feb 2014, 19:42

-2004:
3.3 Reducing Scan-and-Hold Effects
(...)the progressive scan-and-hold aspect of an LCD
requires special treatment to prevent the loss of image
quality compared with scan-and-erase technologies such as
CRTs and plasma displays. This self-erase feature prevents
part of the previous image frame and part of the new image
frame from being simultaneously present on the display.
A Third Generation Timing Controller And Column Driver Architecture
Using Point-to-Point Differential Signaling
http://www.ti.com/lit/an/snla170/snla170.pdf

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: So what refresh rate do I need? [Analysis] [very good on

Post by Chief Blur Buster » 23 Feb 2014, 20:57

hwps wrote:A Third Generation Timing Controller And Column Driver Architecture
Using Point-to-Point Differential Signaling
http://www.ti.com/lit/an/snla170/snla170.pdf
Thanks for the chime in!

I see this paper uses very different terminologies, scan-and-hold versus scan-and-erase.
I interpret it to mean equivalently as:
scan-and-hold display == sample-and-hold display
scan-and-erase display == impulse-driven display / strobed display / flicker / black-frame insertion

There's actually really a complete continuum between the two, as it's really a matter of adjusting duty cycle between the visible frames and the black period between frames (And how much blending there is --e.g. pixel transition effectds, phosphor decay effects, etc) though for new strobe-backlight gaming displays, they act essentially quite squarewave in "hold"/"erase".

_________________

BTW, on another topic but related, tilt artifacts caused by display scanning.
Higher refresh rates have less scan artifacts (tilting artifacts).

Technically, from a motion clarity perspective, it doesn't really matter if it's scanned top-to-bottom or flashed all at once (instantly). The main artifact difference is the tilting effect during horizontal motion (e.g. www.testufo.com/blurtrail at Height->Full Screen on 60Hz LCD or CRT, will often show a tilting-line effect due to the finite speed of the top-to-bottom scan), and compress/stretch effect during vertical motion. A good paper about the tilting effects of scanned display is Charles Poynton's paper, Motion portrayal, eye tracking, and emerging display technology. In addition, he has some illustrations that explains why we need to stick to sequential scanning because of the disjointed-scan effect (stationary tearline effect). He writes about the tilting effect caused by scanning on a LED marquee. This also applies to scanned displays (CRTs and LCDs).
Page 5 of Charles Poynton's Paper wrote:Image
This is another good reason of using higher refresh rates for non-strobed scanned displays -- at double the refresh rate, you get half the tilting effect for a specific motionspeed. I can still see the tilting effect at 120fps @ 120Hz at www.testufo.com/blurtrail in non-strobed mode. All-at-once strobing does not create any tilting effect. That's why you will see the tilt effect when LightBoost is turned off, but the tilt effect disappears when you turn on LightBoost!

Also -- I have had at least two people suggest displays could refresh faster only where the eye is pointed at (e.g. random-access refreshing). This is a very good idea, and definitely should be tried. It may make the ultrahigh refreshrates and/or continuous-motion framerateless displays possible. However, the disjoints in refreshing synchronization will be very problematic for potential artifacts, for these very precise reasons already illustrated in Charles Poynton's paper.

Thusly, to avoid distortion effects, we're almost certainly to be stuck with various synchronized sequential-scanning or all-at-once presentation, for the most artifact-free motion. Otherwise we get disjoint problems caused by multi-scanning or random-access-refresh. Even simple things like interlacing creates venetian-blinds artifacts (Animation: www.testufo.com/interlace). This, therefore, humankind (in the coming decades) may not have much choice but to refresh sequentially, and go for ultrahigh(quad-digit frame rates, e.g. 1000fps@1000Hz or similiar ultrahigh rates) or framerateless continuous-motion technology for the entire display, to simultaneously solve the stroboscopic problem and the motion blur problem during ultrafast motion material (as discussed in earlier pages in this thread). Obviously, these may not be practical today or this decade, but may become more and more practical as time passes.

Conclusion/TLDR:
- Scanning creates tilt effect on motion perpendicular to scan direction;
- CRTs and non-strobed LCD will have the scan tilting effect;
- Higher refresh rates / faster scanning will have less tilting effect;
- LightBoost (and similar strobe backlights) will not have the scan tilting effect;
- Multiscanning will create stationary tear-line artifacts;
- Interlacing will create venetian blind artifacts;
- Anything other than all-at-once presentation can cause motion distortions, in one form or another;
- Sequential scanning is the least-distorting method of scanning, short of doing all-at-once presentation.

Interesting DIY Science on tilt effect of scanning: Load www.testufo.com/blurtrail on an iPad!. Pinch-zoom until the black animation with the moving line, just about fills the whole screen. You will notice the tilting effect of the moving line! Now rotate the iPad to all different 4 rotations, observe the tilt effect. You will see tilting in 2 orientations, and no tilting in 2 other orientiations. At least one apple iDevice I tried has landscape scanning (iPad 2), and a different apple iDevice I tried (Retina iPad Mini) has portrait scanning -- the tilting showed up only in portrait mode! Try this out now. (iPads worked best, since they can run the HTML5 animation at the proper synchronized 60fps@60Hz)
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

Post Reply