Temporal anti-aliasing for computer generated graphics
Posted: 27 Feb 2014, 09:45
What is temporal aliasing and why is it a problem?
Due to the time-discrete nature of our displays, transformation speeds (movement, rotation) less then half the frame rate causes false detail. For example, a wheel rotating at 120 Hz filmed by a 24Hz camera with very low exposure duration speeds appear static, when they should be blurred ('wagon-wheel effect). Without motion blur, computer generated images show aliasing even at 10,000 frames per second. Therefore, computer graphics have to simulate motion blur to reduce it.
Rendering Examples:
Assume we have a 100 Hz display with 2ms persistence and 100% fill factor.
1000x1000 Pixels, 50 degree FoV.
Example 1. Stroboscopic effect:
Movements faster than 1 pixels per frame causes objects to jump.
Setup: Fixed eye, pixel moves at 233 pixels per second from left to right
No aliasing - inconsistent, jumpy movement (yellow=ideal)
Spatial aliasing makes the movement time-consistent, but gaps remain
Directional motion blur reduces gaps, but inconsistency remains
A combined approach is needed.
Example 2: Eye tracking:
Eye tracking needs to be accounted for, especially when the screen has a large FoV, such as head mounted displays. Same setup as Example 1 with motion blur, but now our eyes track the movement of the pixel. Pixel movement is blurred, but should be sharp. Display-static objects have an apparent motion of 233 pixels per second from right to left and suffer from stroboscopic effects.
Example 3. Phantom array effect:
Lower persistence reduces blur during saccades, disabling saccadic surpression and causing perceived backward motion.
Setup: 20 degree horizontal saccade (25ms duration), static pixel (pixel width = 0.05 degree)
blue line = saccadic movement
black line = hold-type blur (non-linear as the eye-velocity changes)
2ms persistence. Up to 66 pixels blur (3.3 degree)
Full persistence. Up to 177 pixels blur (8.8 degree)
Conclusion:
Time discrete rendering causes temporal aliasing. As resolutions increase, impractical frame rates are necessary to avoid it. Spatio-temporal anti-aliasing can converge to an ideal solution, thus eliminating the need for higher frame rates (beyond flicker fusion), but eye movements need to be accounted for.
Limitations:
In practice it isn't possible to deal with temporal and spatial aliasing separately and achieve 'perfect' motion blur. See:
Shinya et al. 1993 / Spatial Anti-aliasing for Animation Sequences with Spatio-temporal Filtering
Fast eye-tracking system is still an unsolved problem for head-mounted displays.
Furthermore, as seen in example 3, the complex relationship between the human visual system and object motion is still an active area of research, and can improve effectiveness and efficiency of motion blur rendering techniques.
Annex: Overview of algorithms.
Navarro et. Al 2010 / Motion Blur Rendering: State of the Art
Due to the time-discrete nature of our displays, transformation speeds (movement, rotation) less then half the frame rate causes false detail. For example, a wheel rotating at 120 Hz filmed by a 24Hz camera with very low exposure duration speeds appear static, when they should be blurred ('wagon-wheel effect). Without motion blur, computer generated images show aliasing even at 10,000 frames per second. Therefore, computer graphics have to simulate motion blur to reduce it.
Rendering Examples:
Assume we have a 100 Hz display with 2ms persistence and 100% fill factor.
1000x1000 Pixels, 50 degree FoV.
Example 1. Stroboscopic effect:
Movements faster than 1 pixels per frame causes objects to jump.
Setup: Fixed eye, pixel moves at 233 pixels per second from left to right
No aliasing - inconsistent, jumpy movement (yellow=ideal)
Spatial aliasing makes the movement time-consistent, but gaps remain
Directional motion blur reduces gaps, but inconsistency remains
A combined approach is needed.
Example 2: Eye tracking:
Eye tracking needs to be accounted for, especially when the screen has a large FoV, such as head mounted displays. Same setup as Example 1 with motion blur, but now our eyes track the movement of the pixel. Pixel movement is blurred, but should be sharp. Display-static objects have an apparent motion of 233 pixels per second from right to left and suffer from stroboscopic effects.
Example 3. Phantom array effect:
Lower persistence reduces blur during saccades, disabling saccadic surpression and causing perceived backward motion.
Setup: 20 degree horizontal saccade (25ms duration), static pixel (pixel width = 0.05 degree)
blue line = saccadic movement
black line = hold-type blur (non-linear as the eye-velocity changes)
2ms persistence. Up to 66 pixels blur (3.3 degree)
Full persistence. Up to 177 pixels blur (8.8 degree)
Conclusion:
Time discrete rendering causes temporal aliasing. As resolutions increase, impractical frame rates are necessary to avoid it. Spatio-temporal anti-aliasing can converge to an ideal solution, thus eliminating the need for higher frame rates (beyond flicker fusion), but eye movements need to be accounted for.
Limitations:
In practice it isn't possible to deal with temporal and spatial aliasing separately and achieve 'perfect' motion blur. See:
Shinya et al. 1993 / Spatial Anti-aliasing for Animation Sequences with Spatio-temporal Filtering
Fast eye-tracking system is still an unsolved problem for head-mounted displays.
Furthermore, as seen in example 3, the complex relationship between the human visual system and object motion is still an active area of research, and can improve effectiveness and efficiency of motion blur rendering techniques.
Annex: Overview of algorithms.
Navarro et. Al 2010 / Motion Blur Rendering: State of the Art