Async Reprojection + DLSS 2/3 + Reflex + GSync = The Holy Grail?

Everything about latency. Tips, testing methods, mouse lag, display lag, game engine lag, network lag, whole input lag chain, VSYNC OFF vs VSYNC ON, and more! Input Lag Articles on Blur Busters.
Post Reply
PeanutArbuckle
Posts: 1
Joined: 30 Apr 2023, 12:14

Async Reprojection + DLSS 2/3 + Reflex + GSync = The Holy Grail?

Post by PeanutArbuckle » 30 Apr 2023, 13:25

First post here, hi everyone!

I've been looking into async reprojection for the past day after seeing a few videos on it. Notably by LTT, 2kliksphilip, and of course Comrade Stinger who has a downloadable demonstration of the technology in the description of his video. All three are linked below. It might be a good idea to check a few of these videos out before reading the rest of the post as they all explain in layman's terms what async reprojection exactly is.

LTT - youtu.be/IvqrlgKuowE
2kliksphilip - youtu.be/f8piCZz0p-Y
Comrade Stinger - youtu.be/VvFyOFacljg

Now, I've run the demonstration myself and I've noticed a few things about how this implementation of async reprojection works. For context I am using a 60 hz monitor.
- With the framerate uncapped, as expected the mouse movement is very smooth and so is just walking around left to right without moving the mouse.
- With the framerate capped to 30 fps, the mouse movement is no longer smooth and walking around left to right without moving the mouse makes models very juddery. This is most noticeable when aligning the mouse to at the CS:GO player model directly and then only using the keyboard to move left and right.
- With the framerate capped to 30 fps with async reprojection, the mouse movement when not moving the character with the keyboard is as smooth as uncapped, however you can see black areas where the frame is not rendered. This can be somewhat mitigated by enabling "stretch timewarp borders" where the black is replaced by a stretched version of the rendered version of the frame. When moving the character left to right as in the previous example, however, it becomes obvious that we are still running the game at 30 fps as the judder still exists.
- However, if we increase the framerate cap to at least match our refresh rate (60 fps - 60 hz) and enable async reprojection + the timewarp fix, then the stretch artifacts become almost unnoticeable and the judder seen when moving the character with the keyboard is also completely eliminated. Not only that, but we should have all of these benefits and the smoother/more precise mouse input that async reprojection is made for. It's essentially 60 fps gameplay with minimal artifacts but with the mouse input lag/feel of an uncapped game (though in reality the "refresh rate" for the mouse input would be whatever is set by the game engine). From what I understand, this should feel more responsive than a 60 fps game without async reprojection.

Now I've been thinking, and this is a topic touched on some of the videos I've linked too, but what if this async reprojection technology was further refined and then used with other technologies like DLSS 2/3, NVIDIA Reflex, and GSync? If async reprojection works best when the fps is at least matched with the monitor's refresh rate, then something like DLSS 2 (or FSR 2.0) can be useful to at least reach that refresh rate on lower-end hardware. Then let's say we added DLSS 3 (or FSR 3.0, soon to be released) on top of that which doubles our framerate. Normally, we would have to deal with increased input lag because DLSS 3 holds one frame in queue to generate a frame between the upcoming frame and the previous frame, but as mentioned in 2kliks' video if mouse input and framerate are decoupled from each other it could be possible to enjoy the benefits of DLSS 2 and 3 without the input lag downside.

It might even be feasible to generate two or more DLSS 3 frames to further multiple fps if async reprojection allows mouse input to be processed by the game at a higher, fixed rate (like maybe 1000 times per second for example). Any additional latency overhead from all the processing could be reduced with something like NVIDIA Reflex, and to my knowledge enabling Reflex when using DLSS 3 even now helps lower latency a bit.

Finally, for those with higher refresh rate monitors who cannot always hit their refresh rates for a certain game, the enabling DLSS 2, DLSS 3, NVIDIA Reflex, and now GSync along with async reprojection would mean that the refresh rate of the monitor is always the same as the game eliminating all judder when moving with the keyboard. Thus, with all of these technologies being used at once, you would in theory have smooth gameplay display-wise thanks to DLSS 2/3 and GSync and smooth gameplay mouse input-wise thanks to async reprojection without any of the compromises/downsides that you normally would get by using these technologies independently. This could be game-changing for single-player games like Cyberpunk, Resident Evil, and so forth.

Imagine the following scenario: You are playing Cyberpunk on a mid to high end rig with a 40-series NVIDIA card. Without any of this tech enabled, at your graphics settings you get around 50-60 fps on your 240 hz, GSync monitor. After enabling DLSS 2 and 3 your framerate increases to 100-120 fps, but now you have increased input lag thanks to DLSS 3. So, you enable async reprojection and the mouse input issue is gone, but now the game is juddery because your framerate is lower than your refresh rate. Next you enable GSync so that your refresh rate always matches your framerate and now the judder issue is solved too. Reflex is also enabled to further lower input lag. End result is that you doubled your fps at basically no cost, and potentially have even better mouse input now than you would if you were playing at native 100-120 fps.

The only caveat I can really think of for this technique is that the DLSS 3 part of the whole process might not be great for competitive FPS games due to the fact that the fake frames don't actually contain real information, just extrapolated information, but async reprojection could definitely be useful, especially for those playing at lower framerates and refresh rates. Obviously, adding async reprojection won't help with the input lag directly related to the number of frames displayed on your monitor, just the mouse input.

I also wonder if this could be applied to input as a whole, meaning both mouse and keyboard input is decoupled from the game's framerate. Perhaps it doesn't matter as much for the keyboard as it does for the mouse but I don't see why it shouldn't be possible. With all the games that are releasing these days with horrible optimization it would be cool to see tech like this that would at least make these games somewhat playable!

Post Reply