[BREAKTHROUGH] 4K 1000fps 1000Hz now possible on today's GPUs via reprojecton

Everything about displays and monitors. 120Hz, 144Hz, 240Hz, 4K, 1440p, input lag, display shopping, monitor purchase decisions, compare, versus, debate, and more. Questions? Just ask!
elexor
Posts: 169
Joined: 07 Jan 2020, 04:53

[BREAKTHROUGH] 4K 1000fps 1000Hz now possible on today's GPUs via reprojecton

Post by elexor » 31 Oct 2022, 19:20

First demo I have seen that uses mouselook reprojection or timewarping

This seems like a nobrainer to me, been thinking about this since the early oculus days.

phpBB [video]


Link to Comrade Stinger's demo in the video description

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

VIDEO mouselook timewarp and positional reprojection (Frame Rate Amplification Tech)

Post by Chief Blur Buster » 31 Oct 2022, 23:25

Blur Busters deems this a breakthrough

That's fantastic that this is being worked on -- mouselook reprojection!

Reprojection is mentioned as part of my Frame Rate Amplification Tech article.

This needs to be a standard Vulkan/DirectX API feature, IMHO.

The graphics drivers can decide how to render the reprojection (e.g. NVIDIA vs AMD vs Intel), or even be overridden by a user hook (e.g. game engine's preferred reprojection algorithm), so that all vendors are free to innovate.

But there should be an industry standard 3D API extensions to help frame rate amplification technologies.

Now game engine makers need to add this.

Cyberpunk 2077 at 500fps 1440p+ should be possible with this technology on today's RTX 4090
Via well-optimized reprojected mouselook, though gametime:photontime needs to be kept jitter-free between the real frames and the reprojected frames, to feel properly smooth. Reprojecting the frames is much lower workload than the original frames, so framepacing differences need to be covered as much as possible. However, this demo seem to already do this -- very cleverly.

Reprojection is common in VR.

It's amazing that reprojection has not been ported earlier to high-Hz monitors. In fact it would help both 2D and 3D -- e.g. reprojection to pixelshift an RTS game to do DOTA2/LOL at 500fps on 500Hz for near-blurless sample-and-hold on today's existing technologies.

Now, adding 6dof reprojection capability (like ASW 2.0) would also increase strafing framerates independently of the game engine's framerate too!

It's oftentimes a 3rd party leads the way to show the software developers how easy this is to do!
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

elexor
Posts: 169
Joined: 07 Jan 2020, 04:53

Re: VIDEO mouselook timewarp and positional reprojection (Frame Rate Amplification Tech)

Post by elexor » 01 Nov 2022, 01:56

phpBB [video]


The guy that made it here. also a newer version with improved player reprojection.

EDIT: oh wow the improved reprojection is much better less jarring artifacts.

ethereal_intellect
Posts: 1
Joined: 02 Nov 2022, 05:11

Re: VIDEO mouselook timewarp and positional reprojection (Frame Rate Amplification Tech)

Post by ethereal_intellect » 02 Nov 2022, 05:21

This is also something that i've been thinking about for a long time - and it might be best to strike now that the iron is hot with dlss 3. My proposal is using reshade into a game and reshade into an obs capture, with obs acting as the "compositor" . There's a couple optical flow implementations that have surfaced recently
https://github.com/JakobPCoder/ReshadeMotionEstimation
https://github.com/martymcmodding/ReShade-Optical-Flow
There's also the possibility to hook python into reshade and maybe add mouse movement prediction for the last 25ish ms of lag that have to exist even on high end monitors - VR definitely uses prediction for this too
https://github.com/dwojtasik/PyHook (way more advanced than just a simple position - but shows the connectability of the new reshade builds)
I'm not yet that advanced at shader code tho, but with the mouse input of reshade it should hopefully be simple enough to have a shader offset the whole view. Eventually that would hopefully use a predicted mouse position instead of leaving the last 25ms on the table - but still could be nice to get started with things.

It would also be beneficial for cloud gaming - and a feature that stadia teased and never released

Arbybear
Posts: 9
Joined: 04 Oct 2020, 01:18

Re: VIDEO mouselook timewarp and positional reprojection (Frame Rate Amplification Tech)

Post by Arbybear » 03 Nov 2022, 00:55

I tried the demo out, it's very impressive. Theoretically could NVIDIA or AMD implement this in their graphics drivers right now to work in any current game? Or would a new API be required?

Kyouki
Posts: 190
Joined: 20 Jul 2022, 04:52

Re: VIDEO mouselook timewarp and positional reprojection (Frame Rate Amplification Tech)

Post by Kyouki » 03 Nov 2022, 03:12

This is really cool, hope this becomes a standard in Graphics APIs.
CPU: AMD R7 5800x3D ~ PBO2Tuner -30 ~ no C states
RAM: Gskill Bdie 2x16gb TridentZ Neo ~ CL16-16-16-36 1T ~ fine tuned latency
GPU: ASUS TUF 3080 10G OC Edition(v1/non-LHR) ~ disabled Pstates ~ max oced
OS: Fine tuned Windows 10 Pro, manual tuned.
Monitor: Alienware AW2521H ~ mix of ULMB/Gsync @ 240hz/360hz
More specs: https://kit.co/Kyouki/the-pc-that-stomps-you

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: VIDEO mouselook timewarp and positional reprojection (Frame Rate Amplification Tech)

Post by Chief Blur Buster » 03 Nov 2022, 20:59

Arbybear wrote:
03 Nov 2022, 00:55
I tried the demo out, it's very impressive. Theoretically could NVIDIA or AMD implement this in their graphics drivers right now to work in any current game? Or would a new API be required?
The game developer typically has to be involved.

So classically it would have to be either:
(A) Reprojection built into the game;
OR
(B) Reprojection APIs built into the OS/driver and the game using the API.

However...

Here's where a small amount of AI can be used to monitor the game initially and autoconfigure middle-layer reprojection unbeknowst to the game engine. In theory, a little bit of AI can automatically recognize the game content + mouse movements + keyboard movements, and auto-train the reprojection parameters:

1. Automatic recognition of game sensitivity
2. Automatic recognition of mouse sensitivity
3. Automatic recognition of the rotational origin of the GPU render (X,Y,Z coordinates that mouselook spins around).
4. Automatically program the reprojector
5. Recognition of Zbuffer/HUDs and avoiding reprojection artifacts on those.

It would only be initial training (e.g. first 5 seconds of game playing) to automatically program the reprojector for that specific game for that specific mouse settings for that specific GPU for that specific game, but it would make it work in existing games.

The problem is there would be no overscan-rendering, unless creative tricks are done that potentially games. Like expanding the game's FOV slightly (works best if supersampled-rendering), and reprojector will crop the FOV, and use only the boundaries as reprojection fill for edges whenever necessary. Many games lets you expand your FOV, and then the reprojection filter can decrease the FOV at the filter-injection stage.

So in theory a 3rd party filter-injection reprojection is theoretically possible. VorpX/ReShade/SweetFX/SpecialK could add reprojection, if it has a little bit of auto-train AI for the first few seconds of gameplay to sync everything up. There would be more artifacts than native game support, but this could be a great way to begin.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

[BREAKTHROUGH] 4K 1000fps 1000Hz now possible on today's GPUs via reprojecton

Post by Chief Blur Buster » 02 Dec 2022, 20:09

LinusTechTips just released a video about this

phpBB [video]


Five stars to LTT for publicizing this.

The very 4000-series GPUs in our machines can now drive 4K at near-retina refresh rates, if game developers added it to UE5 or Cyberpunk 2077. Fully preserving RTX detail level too!

Reprojection artifacts largely disappear when:
- Using sample and hold (no double images); and
- Using original frame rates at least 100fps (no edge-flicker from stutters);

I downloaded the demo. Reprojection artifacts look 1/100th as bad as my original Oculus Rift once converting 100fps->360fps on a 360Hz screen, strobing disabled, and reprojecting from 100 instead of 45. Reprojection would be a huge help for upcoming 240Hz OLEDs!

I was able to reproject to 1440p 280fps and 1080p 360fps from a laptop RTX 2080. The desktop 4000-series can reproject all the way to 4K 1000fps+ using less than 10% of the 4000-series GPU, leaving 90% for doing ultra-high-detail RTX-level Cyberpunk 2077 or UE5 engine graphics.

This would be strobeless ULMB.
Low persistence sample and hold.
Motion blur reduction without strobing.
Real life does not flicker.

The main side effect is a minor amount of 100fps-like motion blur for fast horizontally-moving enemies, but motion blur completely disappears for mouselook, for strafes, and for pans/scrolls. You don't have the double-images or stutter of old VR reprojection. When strobing (like a VR headset or like DyAc) is disabled, then we're instead using reprojection as a strobeless motion blur reduction technology.

Strobing becomes obsolete with reprojection!

No eye searing flicker.
No screen dimming.
Full brightness capability.

We need to convince CD PROJEKT RED as well as EPIC to add reprojection support to Cyberpunk 2077 engine and the UE5 engine.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

yuri
Posts: 46
Joined: 09 Jun 2022, 14:19

Re: [BREAKTHROUGH] 4K 1000fps 1000Hz now possible on today's GPUs via reprojecton

Post by yuri » 03 Dec 2022, 21:12

Wow this seems good, we Hope that this gonna be standard for games !!

User avatar
lyrill
Posts: 385
Joined: 06 Oct 2020, 10:37

Re: [BREAKTHROUGH] 4K 1000fps 1000Hz now possible on today's GPUs via reprojecton

Post by lyrill » 04 Dec 2022, 20:59

ye clearly the testers can tell something is wonk.

and linus has not explained the possible problems himself either.

just from the barebone introductions obviously the peripheral vision is hugely sacrificed if nothing else.

What I meant when I saw rtf repost this on twt is that : even if i'm just static and camping fjell I STill want no sacrifice
and I definitely do not want to lower any pov
as it has become ironclad obvious that any game's max pov is only leaving much to be desired
because they already gimped it
because advancing spherical screen for pc was already largely halted
--I mentioned how whoever laughed at my "not curved" purchase in 2016 in form of the rog 180hz might be ashamed to realize all top esport screens still 2d physically.

Post Reply