Chief Blur Buster wrote: ↑30 May 2023, 17:54Even that amazing desktop reprojection video is actually low-quality Wright Brothers stuff. Oculus (I still prefer that name over "Meta") Rift VR does a better job of reprojection with ASW 2.0.bumbeen wrote: ↑30 May 2023, 06:18wow I had not seen that video, and was very confused until linus showed the bit of the frame floating in black space and then it instantly made sense to me. That is absolutely genius, and it seems like it would even work for multiplayer games like csgo and valorant, although at a higher minimum FPS threshhold that at least matches the tick rate of the game. So 128fps to 1000fps would be quite easy to do here
Good reprojection (ASW 2.0) has fewer artifacts than other persistence-improving techniques such as strobing or detail-level reduction (to increase frame rates).
So for those people coming and not realizing we can reduce display motion blur gigantically in a strobeless way (no flicker like CRT / plasma / phosphor / BFI / strobe). Ergonomic flickerfree method of eliminating display motion blur.
Good, properly done reprojection is not frames that are more fake
Pure strobing and pure frames have their place. Especially for retro games. However, the fake-frame dissmiss-artists forgets to acknowledge the above pragmaticness for modern 3D workflows.
Video compression (Netflix, YouTube, whatever) is often 23 fake frames per second and 1 real frame per second, being that said video compression uses interpolation/prediction systems (e.g. the I-Frame, B-Frame and P-Frame system of many MPEG1/2/3/4 codecs, use interpolation mathematics).
It's not black box like laggy TV interpolation, because the video compressor had access to ground truth (original frames) and the game good reprojector logic has access to ground truth (inputread/geometry/zbuffer/etc).
So, the fakeness of the frame, is pretty much synthetic/academic sematics, considering that all GPU frames can meet the Merriam-Webster dictionary definition of "fake" in some aspects (e.g. artisting a photorealistic UE5 frame using triangles (aka polygons) and bitmaps (that we call textures).
Semantically, rendering a GPU frame is still a synthetic frame.
Reprojection is now potentially a "less fake" way of doing ultra high frame rate, because of what I said above.
Reprojection is simply a different workflow of doing something with fewer artifacts than doing it directly via GPU triangles and textures -- we have to lower detail level to make a frame even more fake, to get higher frame rates -- so we can make frames less fake by using reprojection! It can go artifactless like Netflix fake frames. And if it can be lagless for esports, then why not?
Eliminating reprojection artifacts
The artifacts of reprojection can be diminished further by using a base framerate above flicker fusion threshold (aka 100fps starting frame rate), to eliminate even more reprojection-artifacts.
Sample and hold stutter and blur are exactly the same thing. Low-frequency sample and hold shakes/vibrates like a music string -- And high-frequency sample and hold blurs like a fast music string -- www.testufo.com/eyetracking#speed=-1 (look at bottom of UFO as it changes speed over the course of 30 seconds). By pushing ALL frequencies above flicker fusion threshold (starting frame rate, and all jitter frequencies, etc) well up there, reprojection becomes much more artifactless.
Reprojecting 100fps->1000fps is MUCH fewer artifacts than reprojecting 45fps->90fps.
And you have no double-image effect like Oculus, because it's sample-and-hold (no double-impusling)
Sometimes blur is good. Right tech for right job.
Sometimes we love motion blur. Hollywood Filmmaker Mode. Motion blur has the right artistic tool. But it's bad when trying to simulate a Star Trek Holodeck: More blur than real life. So, there you go. Right Tool For Right Job.
We all love artsy looks like Minecraft or retro games, but we also enjoy real-life-style games (e.g. photorealistic shooters), and that's where displays are throwing bottlenecks at us, or artifacts of attempting higher frame rates at low blur (strobe flicker or low-detail ugliness), which reprojection does fix.
See where I am getting at? Wink, wink.
Large-ratio reprojection is a massive motion blur reduction technology
With large-ratios, reprojection can eliminate as much motion blur as strobing can!
The biggest technological catch is that refresh rates and final frame rates need to be stupendously high.
- 500fps for 2ms MPRT strobelessly
- 1000fps for 1ms MPRT strobelessly
- 2000fps for 0.5ms MPRT stroblessly
This is assuming GtG is practically zeroed out, GtG darn nearly 0ms. OLED is able to do that. With 240Hz OLEDs and 500Hz+ LCDs, and refresh rates still increasing (and 1000Hz tests already done in the lab), we're already very close to 1000Hz displays becoming a reality before the end of this decade.
Now that brings the GPU problem. And reprojection solves the GPU problem.
We blur-busting enthusiasts all want 1000fps UE5.2 or UE6 quality graphics eventually.
Reprojection will successfully get us there, and possibly even on some of today's GPUs (e.g. RTX 4090 can do 1000fps reprojection today; confirmed via tests). 75% of an RTX 4090 can be directed to render original UE-quality frames, and 25% of an RTX 4090 can be directed to convert 100fps to 1000fps in a nigh-lagless manner. NVIDIA theoretically could call this DLSS 4, but it also has to be baked into the game engine (due to need for access to inputread/geometry/zbuffer ground truth, for accurate artifact-free reprojection).
Reprojection can rewind local input lag too! Turning 100fps 10ms rendertime lag, to 1000fps inputreadtime lag. This is done by making sure reprojection has access to current ground truth (inputreads) and past ground truth (fully rendered frames with zbuffer and geometry). The current ground truth (mouselook/strafes/etc) morphs the most recent historical frame to current freshest 6dof inputreads.
With a two simultaneous-workflow GPU pipeline (the original-frames pipeline, and reprojection-pipeline), you can convert a varying-framerate game into a permanent constant framerate=Hz experience that is synchronized to freshest 6dof inputreads (mouselook/strafes/etc).
So a fluctuating 80-120fps gets converted to a perfect 500fps@500Hz via a variable-framerate-compatible reprojection engine that has permanent access to continual 1000Hz 6dof data (even between original-render frames). You get the lagless 500fps VSYNC ON look, from an original fluctuating-framerate stream.
The most important two gods of good blur-reducing stutter-reprojecting reprojection
(A) game timestamp is god (microsecond monotonic counters)
(B) inputread timestamp is god (calculating and executing local player 6dof position to sub-millisecond relative accuracy)
As long as (A) and (B) is met, it successfully becomes a DLSS+ULMB+GSYNC sequel.
This is "DLSS+ULMB+GSYNC+FlickerFree+EyeCare" sequel rolled into one.
- DLSS because reprojection is a frame generation tech (even if lagless)
- ULMB because reprojection also reduces motion blur
- GSYNC because reprojection also can de-stutter a variable input framerate to a constant output framerate
- FlickerFree EyeCare because this is flickerless, strobeless, BFI-less, PWM-less.
So it is simultaneously DLSS 4, ULMB 3, and GSYNC 3 rolled into one!
The reprojection processor on the GPU would run at a higher GPU priority than the original frames pipeline, to ensure the framerate=Hz lagless reprojection experience.
Enemy movements may still stutter, but remember the stutter-to-blur continuum -- 100-200fps is just slightly extra persistence blur, not stutter. So enemy movements might look slightly blurrier than the rest of the screen. Reprojection fixes the most important motion blur (global motion blur of panning / scrolling / turning / strafing), which is why reprojection is the prioritized motion blur reduction technology.
So, reprojection (on ultra-high-Hz nigh-instant-GtG displays) will easily make BFI obsolete for some people, since strobless (Flickerfree! Ergonomic! PWM-free!) motion blur reduction is the Blur Busters Holy Grail of sorts for many of us.
We love strobing, but it's still a (good) bandaid for the fact that computer frame rates is finite/synthetic version of real life, and real life is framerateless.
Since large-ratio reprojection technology is a fantastic motion blur reduction technology;
Next Steps for desktop reprojection
Now, the next steps for reprojection I think should be:
NVIDIA and Epic should collaborate to add lagless frame generation to DLSS 4/5 or ULMB3 (via reprojection) to a future Unreal Engine 6.
(or Intel+XeSS, or AMD+FSR, if they have the skill to beat the fabulous NVIDIA gorilla to this punch)
I even told this suggestion NVIDIA directly (to Seth Schnieder during past emails / past meeting), that they need to consider this. I could tell they knew what I was talking about -- there was no confusion. I have no doubt the engineers at NVIDIA are smart. Let's hope.
There is enough computing power in an RTX 4090 to do 1000fps reprojection, and NVIDIA could use the extra sales that reprojection can cause. Even the budget gamer who can only afford a $400 GPU is more likely to buy an RTX 4060 if it can do DLSS4 reprojection to turn 20fps games to 60fps (with some artifacts), while the enthusiasts like me will use an RTX 4090 to turn 100fps RTX ON to 240fps-540fps.
If it's even more stomachable to the beancounters at GPU manufacturers, DLSS4 with reprojection could debut during the 5000-series instead. Okay. I can certainly understand the business case (I'm a business too), even if the users will scream holy hell about not getting easy reprojection on older GPUs, like that famous LTT reprojection video is extolling about.
Heck, I can do Oculus reprojection on an old GTX 1080 GPU, converting Oculus Rift 45fps to 90fps. But, I would still implore NVIDIA to introduce DLSS4 (at least forward reprojection) retroactively to RTX 4000 series.
I feel that NVIDIA does not need to fear sales and revenue reductions -- reprojection will help increase GPU sales by giving more 60-120fps experiences to more mainstream people (including casual gamers), while giving the high end gamers the 500+fps strobeless blur reduction experience.
It can be basic for now, e.g. slightly laggy reprojection that doesn't have rendertime-latency rewind capability at first. The complete ULMB+GSYNC+DLSS combined sequel that is lagless-ready for esports, can come later (e.g. DLSS5 or DLSS6). But the starting pistol of reprojection needs to begin soon -- like 2023. Definitely 2024 at the latest for one of the flagship game engines.
There is little demand now only because nobody knows this as well as Blur Busters does.
Remember our LightBoost HOWTO? Remember our G-SYNC 101? We helped the sales years before many marketing people at NVIDIA realized this. This is BlurBusters' "Big Hit #3" -- we are even about to crowdfund a third party reprojection processor for a desktop game engine. But I'd rather see NVIDIA beat me to the punch. Will be glad to see NVIDIA lift all refresh rate boats, win win even if my project becomes a throwaway project. So, NVIDIA, you heard me. I consider this potential Hit #3 of the mid-2020s after LightBoost HOWTO and G-SYNC 101.
Why am I loud? Big boom of OLED
I've seen the Wright Brothers version of this in action. It's that impressive.
I am actually, in real life, witnessing with my eyes, the tantalizing simultaneous ULMB+GSYNC+DLSS+EyeCare+FlickerFree rolled into one, when framerate=Hz reprojection is executed on a high-Hz OLED.
So, NVIDIA needs to address the boom of OLED, increase GPU sales to OLED purchasers. With gamers kind of frustrated at NVIDIA, this technology would be a major appeasement. Framerate progress should no longer be held back -- RTX ON pathtracing and triple-digit frame rates is currently mostly mutually exclusive -- but it does not need to be.
Today, 240fps reprojection for pathtraced content on today's 4000 series!
- ASUS 27" desktop ROG Swift OLED PG27AWDM (shipping 2023)
- LG 27" desktop UltraGear OLED model 27GR95QE-B (now shipping)
- LG 45" ultrawide desktop UltraGear OLED model 45GR95QE (now shipping)
- Corsair 45" ultrawide desktop Xeneon Flex model 45WQHD240 (shipping 2Q 2023)
- Dough 27" desktop Eve Spectrum OLED (shipping 2023)
- Acer 27" desktop Predator X27U (shipping 2023)
- Acer 45" ultrawide desktop Predator X45 (shipping 2023)
- MSI ultrawide desktop 491C QD-OLED (shipping 2023)
Tomorrow, 1000fps ULMB3 replacement.
This is the motion blur reduction and stutter-killing technology of the 2030s.
Strobing will remain important for a long time (especially today's LCD, plus also on retro content that is ruined by frame generation). That being said, framerate=Hz on OLED means that 4ms MPRT on OLED looks better than 2ms MPRT strobed, since the extra biasing factor (The simultaneous "ULMB+GSYNC+DLSS+FlickerFree+EyeCare" effect) puts a gigantic elephant-weight on the motion quality of strobeless motion blur reduction.
The Business Case For Reprojection on OLED
Why is reprojection vastly more effective on OLED?
Why is it potentially very profitable for GPU vendors?
And hey, eventually, even VR games will be able to get all that RTX ON goodness, with no framerate compromise! With the new reprojection algorithms that convert a varying framerate input, to a constant framerate=Hz output. So these innovations can be ported back to VR markets too, lifting all Hz boats, and expanding beyond the sales-limiting esports silo, too.
- OLEDs have massive upgradefeel appearance with reprojection algorithms
120-vs-240fps is more visible on OLED than LCD, more mainstream people will notice.
---- OLEDs on desktop usually dont have BFI
Reprojection is a successful alternative
---- OLEDs stutters are more visible at 60fps, requiring 90fps+ to look good
The ultra fast GtG of OLED makes 60fps more stuttery on OLED than LCD. Reprojection fixes this by going to higher framerate. Some people get motionsick OLED stutters, or from from stutters of turning RTX ON, so having RTX ON at framerate=Hz, in a lagless way via rendertime-rewind-to-inputtime, I believe, visually creates a lagless motion nirvana that leads to more sales all across the board.
---- OLEDs does not have much brightness headroom
Reprojection does not degrade brightness
---- OLEDs has fantastically fast pixel response and looks great at framerate=Hz
240Hz OLED has clearer motion than most unstrobed 240Hz-500Hz LCD
Note: This forum post is about to be converted to a Blur Busters article shortly; I often use my forums as a drafting table for future articles. More than half of www.blurbusters.com/area51 are rewritten/polished/curated versions of my past forum posts here. I intend to fire off a race between all the GPU vendors, even though I am rooting for NVIDIA's brilliant engineers to do it first, helped by their brilliant robot arm pursuit camera
Thank you for this throughly detailed post. I've learned so much here and just want you to know your work is not unnoticed or underapprecited.
Hopefully oleds with better motion come sooner rather than later. Despite my desire to get a new monitor since my 2540 is dying I really don't want this push towards better motion Clairity to die out to some other fad.