DLSS4. We need proper tests for motion quality of frame generation

[RonsonPL]

Hi.

Please excuse the bad and chaotic form of the text.
I hope you endure as the meaning matters, not the form. Some points I make may also feel weird/wrong/unnecessary, but I assumed it's better to mention everything just in case.

If someone who's good at writing could upgrade this into a proper testing guide/recommendation, can help, I'll appreciate it. I am not able to phrase and format my thoughs so laymen don't freak out after reading 10%. You know how little knowledge people have about the motion. And how people hate reading badly written walls of texts. But if someone could help making a well written guide, then maybe it could be used to send where it matters. Good youtuber's Discord channel for example.
Somewhere where it can make a difference.



Here is what I think is needed to conduct the test of DLSS4's multi-frame generation technology (and any other interpolation methods)

1. Who can performs the tests:

- The test should be performed by a knowledgable person. One that not only knows well how proper motion looks on the best strobed LCDs, but also who played a lot of games with crystal clear motion,

- The person should know what to look at and to not waste time, for example, on looking at the moving object without tracing it with his eyes. I've seen youtubers moving the mouse left and right while staring at the center of the screen. I've seen people who got used to blurry motion so much, they don't even look at anything before their mouse movement stops.

- Healthy eyes. The tester should be able to easily track ufo motion in 120Hz Blurbuster's UFOtest at 1920 pixels per second setting at 1080p res (full screen).
Minority of people have issues and are unable to move their eyes fluidly and therefore won't be able to see the moving objects on screen clearly. We need the most rigoristic people for these kinds of tests. We need an opinion from someone who is accustomed to watching a sub 2ms persistence on a strobed TN monitor or 500Hz OLED. Nothing should be left to personal opinions. If someone is OK with VHS level of picture quality in fast motion, that's his right to have such an opinion, but we don't want the test results to be useless for more demanding gamers. I've seen people saying the image in motion is absolutely perfect.. on 240Hz OLED monitors. Just becasue the difference to their old VA LCDs is so big.

Testing methodology and suggestions:

- the base framerate should be 120fps, so the test can be performed also with human eye. Tests for 60fps source would also be very useful, for example for games which regardless of settings cannot run above 100fps. But currently DLSS4 doesn't support more than x4 frame gen, and with target being 240Hz, the frame persistence will make it hard to judge by humans and since there's no hardware capable of 240fps capture, it would require suitable hardware, like high-speed camera etc.

- The test should include a game which can work properly without TAA, to better understand the difference in clarity of the picture in fast motion without TAA messing with it.
Most modern games fail - they don't even allow to play with clear motion, due to forced TAA and severe issues related to undersampling when TAA is disabled. Not even 16K resolution can fix the pixellation it introduces.

This subreddit
https://www.reddit.com/r/FuckTAA/
can provide guides on how to manually disable TAA even if the game developer didn't offer the official option for it.

Here's the full list of the games which currently support DLSS4. I'm not sue if even one qualifies for the tests. I hope so. If not, it's better to see what MFG does to image in fast motion even with the horrible pixellation resulting from disabled TAA.

A Quiet Place: The Road Ahead
Akimbot
Alan Wake 2
Aunt Fatima
Backrooms: Escape Together
Bears In Space
Bellwright
Crown Simulator
D5 Render
Deceit 2
Deep Rock Galactic
Deliver Us Mars
Desordre: A Puzzle Adventure
Desynced: Autonomous Colony Simulator
Diablo 4
Direct Contact
Dragon Age: The Veilguard
Dungeonborne
Dynasty Warriors: Origins
Enlisted
Flintlock: The Siege of Dawn
Fort Solis
Frostpunk 2
Ghostrunner 2
God of War Ragnarok
Gray Zone Warfare
Ground Branch
Hitman World of Assassination
Hogwarts Legacy
Icarus
Immortals of Aveum
Indiana Jones and the Great Circle
Jusant
JX Online 3
Kristala
Layers of Fear
Liminalcore
Lords of the Fallen
Marvel Rivals
Microsoft Flight Simulator
Microsoft Flight Simulator 2024
Mortal Online 2
Naraka: Bladepoint
Need for Speed Unbound
Outpost: Infinity Siege
Pax Dei
Payday 3
Qanga
Ready or Not
Remnant 2
Satisfactory
Scum
Senua's Saga: Hellblade 2
Silent Hill 2
Sky: The Misty Isle
Slender: The Arrival
Squad
Stalker 2: Heart of Chornobyl
Star Wars Outlaws
Star Wars Jedi: Survivor
Starship Troopers: Extermination
Still Wakes The Deep
Supermoves
Test Drive Unlimited Solar Crown
The Axis Unseen
The Finals
The First Descendent
The Thaumaturge
Torque Drive 2
Tribes 3: Rivals
Witchfire
World of Jade Dynasty

- the test should be done on 480Hz OLED screen

- a joypad controller should be used and using mouse should only happen if the game is confirmed to support 8000Hz polling rate correctly, and 8000Hz mouse is used

- It may be useful to check the game before proceeding with the tests on 480Hz OLED. For example: at 120Hz with v-sync ON, in strobed mode on a good LCD with proper backlight strobing implementation. And of course with Hz=fps and single strobe, not double strobe.
It would be useful, cause the tester could practically check for issues with motion, like camera panning bug or microstutter, before the MFG is turned on. Just to be sure everything works as intended. Knowledgable person performing the tests should detect issues within a few seconds, as it's very easy to see with the naked eye. Whenever the perfect motion clarity gets spoiled it's instantly obvious and nobody needs to log frametimes and check for spikes. Relying on software is not only difficult and more time consuming, but it can be misleading and not paint the correct picture when analyzed by just the numbers and values.

This should also allow to verify if all motion-ruining techniques are truly disabled. Sometimes in-game motion blur switches or TAA disabling methods won't toggle before the game is restarted.
Regarding framerates and performance, verifying that the games is able to run at constant 144fps with v-sync enabled, should qualify the game for 120fps>480 interpolation.

- the tests should include at least two motion types and preferably various detail on screen. That's because frame generation can work better on constant, easily predictable motion, usually found on top-down and side scrolling games. If no side-scroller or top-down game qualifies, a simple type of move can be simulated by strafing in front of something and/or looking down.


In terms of most difficult things for the FG, usually those can be found in game's vegetation like tall and dense grass, especially if it's being moved by the wind in non-predictable way.
Since DLSS4 relies on AI, it is likely to have varying rate of success depending on the nature of the motion. The things which are more common will likely be used for training, although I'm not sure if DLSS 4's AI model is so sophisticated. It has to be done with its work in miliseconds so maybe not.


Again, feel free to help me improve this. Post suggestions on how I can turn this wall of text into a readable guide, if you have any.


Why do I post this thread:
Sadly, Hardware Unboxed made a video with the tests.
They
- failed to use an OLED faster than 240Hz
- therefore, didn't conduct any low persistence tests at 480Hz
- didn't even mention the aspect of motion clarity in their video. Like it was completely not important. Meanwhile I think it's actually the only important part of frame generation. I can live with 60Hz strobe/BFI. I'd rather play with horrible flicker than use frame gen with 60fps source.

[Chief Blur Buster]

- the test should be done on 480Hz OLED screen

One of the most important lineitems.

This is my only reply and call-out at this juncture.

[Discorz]

- The person should know what to look at and to not waste time, for example, on looking at the moving object without tracing it with his eyes. I've seen youtubers moving the mouse left and right while staring at the center of the screen. I've seen people who got used to blurry motion so much, they don't even look at anything before their mouse movement stops.

Doing just the eye tracking would be unfair as well because it is not the only way people use their eyes. The fact that people naturally tend to stare through motion instead of track it suggests there are other, perhaps more common use cases.

As a reminder refer to this illustration - when we track moving objects in real life we see perfect motion clarity, but also when not tracking we see perfect motion blurrity. Ideally when testing for motion performace, either with frame gen or not, both cases should be taken into consideration. Defective frames of frame generation will translate to all scenarios.

• Tracking = testing for motion clarity and potential errors
• No tracking = testing for motion blurrity/smootheness and potential errors

Frame generation is not just supposed to be a motion blur-reducing feature, but also a smoothness/blur-increasing and latency-reducing feature. In fact FG at this moment is primarly considered as a motion-smoothing feature. So staring instead of tracking is a fair way to test the fluidity. Thats why AMD called their FG "Fluid Motion Frames", since we don't gain the identical benefits as with raw frame rate.

A future form of lagless reprojection has a massive potential to kill not two but multiple birds with one stone - no blur, no phantom array or stroboscopic artifacts, no flicker to reduce blur, no latency, more smoothness. We are yet to see at what artifact cost. In the meantime we'll wait for 1000+ Hz.

Interestings: There is actually a third test case for BFI - moving eyes on stationary objects where flickering introduces stroboscoppic effect to already ideal persistence, therefore feels less smooth and more stuttery. Noone ever mentions that. But that's perhaps another topic.

[1000WATT]

- the test should be done on 480Hz OLED screen

One of the most important lineitems.

This is my only reply and call-out at this juncture.

Why?
Are we testing frame generation or the monitor?

[1000WATT]

Oh, I know how chief answers. There will be 4 volumes of "War and Peace" here now.
I'll change the question. Wait a bit. Don't answer yet.

[1000WATT]

I don’t recognize you at all, you are ready to entrust so many things to the precision of the brain’s perception, to the eyes, to the movements of the hands.

- the test should be done on 480Hz OLED screen

- a joypad controller should be used and using mouse should only happen if the game is confirmed to support 8000Hz polling rate correctly, and 8000Hz mouse is used

- It may be useful to check the game before proceeding with the tests on 480Hz OLED. For example: at 120Hz with v-sync ON, in strobed mode on a good LCD with proper backlight strobing implementation. And of course with Hz=fps and single strobe, not double strobe.
It would be useful, cause the tester could practically check for issues with motion, like camera panning bug or microstutter, before the MFG is turned on. Just to be sure everything works as intended. Knowledgable person performing the tests should detect issues within a few seconds, as it's very easy to see with the naked eye. Whenever the perfect motion clarity gets spoiled it's instantly obvious and nobody needs to log frametimes and check for spikes. Relying on software is not only difficult and more time consuming, but it can be misleading and not paint the correct picture when analyzed by just the numbers and values.

Instead of a mouse, a script that moves the cursor exactly a certain number of pixels per second in the directions we need.
Or a game benchmark.

benchmark
FG off, 120 fps OBS record on a good codec.
We get a picture close to the reference one.

FG off, We lower the frequencies until the video card renders 30 fps.
FG on, 120 fps OBS record on a good codec.

Let's compare the blur.



Or are we testing the monitor?

If we assume that we managed to increase the frame rate exactly 4 times.
The only thing that a 480 Hz monitor will give is the ability to rotate the camera faster as long as gtg mprt lasts.
And if you compare monitors - 120 Hz at 120 fps and 480 Hz at 480 fps, the difference is obvious.
But will the quality of 1 frame
Initially from 30 fps, interpolated x4
Be different from the quality of 1 frame
Initially from 120, interpolated x4

These are not suggestions for testing or criticism. I don't understand you.

like camera panning bug or microstutter,

Ok, we've made sure that the game at 120 fps has no problems. For example, you rotated the camera at 1920 pixels/second.
or any speed that's convenient for you, as long as the picture is clear.
You switch to 480 fps, but you won't use 1920 pixels/second because there won't be any difference, you'll speed up the panning speed, let's say up to 2560 pixels/second.
Where's the guarantee that panning won't fail and won't cause microstutter because of the game engine, which will have to unload and load a larger amount of data per second? And you'll incorrectly decide that fg is to blame.

[RonsonPL]

Frame generation is not just supposed to be a motion blur-reducing feature, but also a smoothness/blur-increasing

I know, but everyone can properly judge the smoothness of the image. More frames = more smooth. This should not be a problem with interpolation.
I understand all you wrote, but I think it's mainly focused and marketted as "smoothing tech", it's likely that it will change once more people are exposed to perfect motion clarity and such marketing will start to be viable. If someone started to promote mainly motion clarity now when marketing frame generation, it would be destined to fail even if the execution was perfect.
Most people don't know what clear image in motion even is. They will switch to 8ms peristence on OLED 120Hz screen, jumpting from 60Hz VA LCD and they will laugh at this idea. They'll say "what do I need it for? My OLED already is perfect at motion, it looks superb at 120fps".
So when AMD markets it as "fluid motion", they're talking in language gamers can understand.

And there's also the matter of source. 24 or 30fps source won't ever look good in fast motion even with interpolation, cause there's not enough data. Making it look smoother is what FG can do, but making it clearer in motion won't work because there's not enough information for unpredicable, fast motion, to avoid artifacts and errors.

And here again. The mainstream will be more interested in this area - interpolating from lower framerates.
For example, to play fully path traced games where even 5090 is unable to reach 30fps at 4K res.
People interested in FG for HFR sources, like a game that already runs at steady 120fps, will be naturally more interested in the frame persistence aspect of it.


About the stroboscopic effect and testing without tracking the movement with your eyes. I can imagine it may be important, but since I wasn't able to witness low persistence on sample and hold display yet (never seen OLED 480Hz in person) I assume it will remain a mess. I made my tests over 20 years ago, tried 60fps/60Hz, 85, 120 and then 170. The stroboscopic effect's nature changes, but it's still a mess. I may be wrong, but I think even at 480Hz, it will still not matter much, cause it will still be a mess. Maybe 2000Hz could start fixing it. Maybe it can be done better by using superfast eye tracking and motion blur.

I don’t recognize you at all, you are ready to entrust so many things to the precision of the brain’s perception, to the eyes, to the movements of the hands.

People who know about moving camera methods won't need any "testing guidelines". I'm talking about motion newbies. People with youtube channels, hardware forums/sites etc.

Of course it can be better with moving camera tests etc.
Same goes for control methods. Not everyone knows how to use/make scripts. Surely many youtubers don't and I'd like to have as many chances to find someone capable of making the test as possible. Can be even some random guy on some forum. Your script idea should be an option in the hypothetical "guide", though. Mostly for the tests done with proper setup and equipment.

The recording via OBS won't work, because FG starts to look acceptable with the source framerate between 60 and 120fps, with most opinions I saw so far being above 70, 80fps or higher. Testing the motion quality in a FG made out of 30fps source could only end up with a recommendation with very predictable and steady motion, like side-scrolling games (at least for what makes the static backgrounds)

But will the quality of 1 frame
Initially from 30 fps, interpolated x4
Be different from the quality of 1 frame
Initially from 120, interpolated x4

There could be no difference in predictable, steady and slow motion.
But in fast, chaotic motion, there will probably never be a FG method that could "guess" correctly what should be in-between the frames. 30fps is simply not enough motion data.
Let's have an example of a future racer type of game. One frame - you are in front of a bridge. Second frame, you are already past the bridge. How would the alhorithm be able to guess what's below the wooden planks of the bridge? There's no data here. The movement was too fast.
There will be diminishing returns, of course, but the higher framerate the source has, the better the FG's picture quality.

Ok, we've made sure that the game at 120 fps has no problems. For example, you rotated the camera at 1920 pixels/second.
or any speed that's convenient for you, as long as the picture is clear.
You switch to 480 fps, but you won't use 1920 pixels/second because there won't be any difference, you'll speed up the panning speed, let's say up to 2560 pixels/second.
Where's the guarantee that panning won't fail and won't cause microstutter because of the game engine, which will have to unload and load a larger amount of data per second? And you'll incorrectly decide that fg is to blame.

I used the pixels/s only for the UFOtest website.
About the latter part - there's no guarantee, but making sure it's OK at lower framerates is a good start and if it's not, the tester can focus on finding the problem here already.

[1000WATT]

1. Who can performs the tests:
- The test should be performed by a knowledgable person. One that not only knows well how proper motion looks on the best strobed LCDs, but also who played a lot of games with crystal clear motion,

We need the most rigoristic people for these kinds of tests. We need an opinion from someone who is accustomed to watching a sub 2ms persistence on a strobed TN monitor or 500Hz OLED.

I don’t recognize you at all, you are ready to entrust so many things to the precision of the brain’s perception, to the eyes, to the movements of the hands.

People who know about moving camera methods won't need any "testing guidelines". I'm talking about motion newbies. People with youtube channels, hardware forums/sites etc.

The translator is probably mistaken.
But if not.

Or the testing work is taken on by a Person according to your criteria with 5 subscribers on youtube.

Or we create a methodology and tools in a ready-made version. And we hope that a person with 1,000,000 subscribers will be able to read such a methodology.

Knowing in advance that 90+ percent. Either they won't read it or they won't understand the methodology.
Example.
(Repeat steps 8-9 until the Sync Track looks like a horizontal ladder.
It may take a few attempts before you capture a pursuit camera photo accurately.
When done accurately, the Sync Track is successfully captured as follows:)

[1000WATT]

The recording via OBS won't work, because FG starts to look acceptable with the source framerate between 60 and 120fps, with most opinions I saw so far being above 70, 80fps or higher. Testing the motion quality in a FG made out of 30fps source could only end up with a recommendation with very predictable and steady motion, like side-scrolling games (at least for what makes the static backgrounds)

There could be no difference in predictable, steady and slow motion.
But in fast, chaotic motion, there will probably never be a FG method that could "guess" correctly what should be in-between the frames. 30fps is simply not enough motion data.
Let's have an example of a future racer type of game. One frame - you are in front of a bridge. Second frame, you are already past the bridge. How would the alhorithm be able to guess what's below the wooden planks of the bridge? There's no data here. The movement was too fast.
There will be diminishing returns, of course, but the higher framerate the source has, the better the FG's picture quality.

Let's eliminate people's opinions and terms like "fast" and "slow" movement.

Let's have an example of a future racer type of game. One frame - you are in front of a bridge. Second frame, you are already past the bridge. How would the alhorithm be able to guess what's below the wooden planks of the bridge? There's no data here. The movement was too fast.

1. the car speed will be 30 km\h FG off 30fps
2. the car speed will be 120 km\h FG off 120fps

If you don’t see the bridge in the first option, you won’t see it in the second either.
The bridge will either simply disappear.
or there will be artifacts. If 1 frame with a bridge. Will be among 30 frames.
And accordingly 4 in 120 frames

[RonsonPL]

The translator is probably mistaken.
But if not.

OK. Sorry for the confusion. So yes, we do need perfect tests made by knowledgable person. But it's very unlikely we'll get that anytime soon, unless Chief helps us all directly or indirectly.

I'll take tests made by a random forum user or youtuber rather than having nothing. I sure would love to get proper tests, but the MFG requires a 5xxx card and how many people who are knowledgeable have these cards now or will have within next few days? Would be better to get some info now instead of waiting for good tests in October

This thread is just an idea. We can even make two different ones, targetting newbies and "pros", you can even make your own, different to what I suggested here. The only thing that matters is the goal - to learn if MFG is already good enough to battle sample and hold motion blur at the levels comparable to good strobing/CRT simulations. It's obvious it won't be equally good even aside from artifacting, but for people like me, who can't stand the motion blur even on 120Hz OLED TV running a 120fps game in sample and hold mode, it's the most important question this year. Whatever works to get the anwers, man.

1. the car speed will be 30 km\h FG off 30fps
2. the car speed will be 120 km\h FG off 120fps

I'm afraid I don't understand what you mean here.
The test would be done at the same "car" speed, let's say it' 600km/h
at 30fps source, it will just be a frame before the bridge, and then a frame after the bridge. At 120fps source, it will have the frame before the bridge, a frame at the start of the bridge, a frame at the end of the bridge, and a frame after the bridge. Much more data for FG to interpolate something decent.
At least if motion blur in game is disabled, cause for some reason even in future racing games nowadays, it's being enabled by default.