Another software strobing implementation on a 3D engine

Talk to software developers and aspiring geeks. Programming tips. Improve motion fluidity. Reduce input lag. Come Present() yourself!
fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: Another software strobing implementation on a 3D engine

Post by fuzun » 25 Jan 2018, 19:03

Chief Blur Buster wrote:Fantastic! For me, I don't bother, I just use a fixed interval timer, but a badness trigger is interesting!

You could even do it on a per pixel basis. But that is waaaaay overkill.

Remember, for defeating the LCD voltage balancing (inversion) built into monitors, some monitors never burn in, others burn in slowly, and some do burn in fast. So you need a configurable modifier for phase switch velocity (e.g. 2x more frequent, 2x less frequent, etc), even in the worst case scenario, you dont need to phase swap more often than once a minute (so slow down your badness algoritm for those burnin-resistant monitors). So because of this, I never bothered doing an advanced guess of phase-swap necessity. But that can help make phase swaps less frequent!

Also, clever algorithms (e.g. two 50% opacity frames instead of two 100% opacity frames) can eliminate the "bright" flicker of phase swaps done via the repeat-frame technique. You can also for example mathematically calculate opacities, like if you need to flickerlessly convert frame-dark-dark into frame-dark-dark-dark (A 33% lengthening of a cycle), you instead do frame-33%frame-dark-dark. Basically a dark duplicate frame of one-third brightness.

The goal is make sure same number of photons are hitting human eyeballs per BFI cycle, even when the BFI cycle is varying (due to a phaseswap). That reduces flicker of random BFI shortenings/lengthenings.

In other words: Instead of trying to reduce frequency if phase swaps, why not simply make phase swaps invisible instead? Simpler math and easier for other code maintainers.

It maintains the same average number of photons hitting the eyeballs. This algorithm, IMHO, is more useful than complicated "badness" algorithms (useful as they are): Remember, keep it easy for other programmers.

Or keep both (badness algorithm AND phaseswap flicker reduction).

Also, try to detect missed frames (e.g. double vsync) and keep in phase with the monitor, not the game frame. Otherwise you're out of sync, burn in is occuring when you think it is not. Done via extrapolation from microsecond clock to guess correct phase to resync after accidentally missed frames (double VSYNC's, system freezes, etc). Basically, freeze interval divided by known rolling-average vsync interval (remember: when updating a vsync interval estimate, discard the outlier odd vsync-intervals to prevent mucking-up vsync guess). Dividing the freeze interval (get microsecond timestamp right after return of next blocking pageflip call) by the known vsync interval, returns a nearly-integer, and gives you a very good guess of what voltage inversion phase the monitor is currently in. Declare good number if near integer (within 0.1 of being an integer). Ignore if result doesn't look integer. If good, then if odd number, you are in sync, if even number, update your phase flag. (Don't need to phase swap at that instant). Other formulas may be better but, this is the easiest lowest common denominator for guessing the monitor inversion polarity after a long system freeze.

For the burnin prevention "phase-swaps":
My personal opinion in order of priority, IMHO, is:

- Phase swap on fixed integer trigger (user configurable) - maybe even just simple count of unbalanced frames
- Phase swap flicker elimination algorithm - simple opacity (alpha) formula
- Strong Phase resync logic after system freezes.
- More advanced phase swap algorithms (e.g. more complex badness factors).

I am very glad you confirm this solved the software BFI burn in problem.
Wow, those are some quality suggestions. Thank you!

Lowest interval is better for burn in prevention but it also causes (fake) micro stuttering. Sometimes it is not worth to trade because this may become annoying. But your suggestion to make swapping part seamless by showing frames with predetermined opacity is interesting and definitely needs a try.

The main goal of badness thing is actually dynamically finding an optimal timing for swap algorithm to kick in. Because a fixed interval might not provide the best result. But this might not be needed because of the opaque frames technique.
Chief Blur Buster wrote:... you need a configurable modifier for phase switch velocity (e.g. 2x more frequent, 2x less frequent, etc), even in the worst case scenario, you dont need to phase swap more often than once a minute
There is a interval cvar for controlling phase switch in seconds level. I mean if a person wants 2x more frequent then he can lower the interval to half. Is not this the same thing or I am missing a point? And my AUO B156HAN panel burns in a few seconds(60hz to 108hz overclocked. Maybe that is why but there should be support for people who use overclocked panels as well). Is it really enough to swap once each minute?

For Badness,

n -> Percentage of ((+) Phase Normal - (+) Phase Black) out of total (+) Phase frame count. [Can be 0 to 100]
l -> Percentage of ((-) Phase Normal - (-) Phase Black) out of total (-) Phase frame count. [Can be 0 to 100]

What do you think about this table: (N and L are both negative but shown as positive)
Image

The table consists output of "badness (decreases with brightness output, increases with phase unbalance)" function on several different values. Considering it is a logarithmic function, it can be seen from the table that 1, 0 is almost e^4 times better than 48, 0. And 0, 0 is better than 1, 0 by infinite times. Do you really think that the difference between 0,0 - 1,0 is infinite times bigger than the difference between 1,0 - 48,0?

Also is not it strange that 49-51 or 1-99 are equal to 50-50. Considering it decreases with brightness output and increases with phase unbalance, is this possible?


If I can make it seamless by using non opaque frames, badness will only need to depend on burn in inducing factor/phase unbalance. And that will make the job a lot easier.

Assuming that there will still be burn-in issue, function like this will be needed to avoid coding a lot of if statements. But priority of developing it will not be high because since the switching will be seamless, switching can occur much more frequently. So that automatic phase unbalance detection algorithm that is connected with this badness function will not need to kick in most of the time. Maybe if the opaque frame method make it 100% seamless, there won't be need for this function at all, just like you point out.

Chief Blur Buster wrote:Also, try to detect missed frames (e.g. double vsync) and keep in phase with the monitor, not the game frame. Otherwise you're out of sync, burn in is occuring when you think it is not. Done via extrapolation from microsecond clock to guess correct phase to resync after accidentally missed frames (double VSYNC's, system freezes, etc). Basically, freeze interval divided by known rolling-average vsync interval (remember: when updating a vsync interval estimate, discard the outlier odd vsync-intervals to prevent mucking-up vsync guess). Dividing the freeze interval (get microsecond timestamp right after return of next blocking pageflip call) by the known vsync interval, returns a nearly-integer, and gives you a very good guess of what voltage inversion phase the monitor is currently in. Declare good number if near integer (within 0.1 of being an integer). Ignore if result doesn't look integer. If good, then if odd number, you are in sync, if even number, update your phase flag. (Don't need to phase swap at that instant). Other formulas may be better but, this is the easiest lowest common denominator for guessing the monitor inversion polarity after a long system freeze.
It is necessary to detect missed frames, but it is not easy I think. Using microsecond clock can only yield predicted results. And if this result is used, it can even break a working strobing frame sequence. I did not test this method before but it seems by your point that results are quite good. So I will also probably work on that.

I am also thinking that maybe there is some kind of public or private API in graphics driver for this? Instead of bothering with this, using a handy API would be much better.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Another software strobing implementation on a 3D engine

Post by Chief Blur Buster » 25 Jan 2018, 20:52

fuzun wrote:Is it really enough to swap once each minute?
Burnin-speed varies a lot for of defeated voltage-polarity-rebalancing (inversion algorithm) in LCD panels.

I have tested flicker burn-in tests ( http://www.testufo.com/flicker ) on about 20 displays. Some burn in really fast (few seconds) and some burn in after one hour. Some never burn in (but you never know about several days of use). Panels not designed for it (e.g. intentional overclocks) do often tend to burn in faster. Just make it configurable over a humongous range.

To generically make that TestUFO test safe, I just generically defaulted to once every 1800 refresh cycles and as a result, I have few complaints about that.
fuzun wrote:Maybe if the opaque frame method make it 100% seamless, there won't be need for this function at all, just like you point out.
It won't be always 100% seamless. But it will greatly reduce the flicker/stutter appearance of phase changes.

It'll look more subtle if the gamma is balanced (linear gamma). You may need an optional gamma modifier to compensate -- for displays where RGB(128,128,128) is very different from being half the brightness of RGB(255,255,255) -- basically a gamma compensation. Also certain displays with very bad overdrive may have sudden ghosting-amplification during that blended frame, but that won't be all displays.
fuzun wrote:I am also thinking that maybe there is some kind of public or private API in graphics driver for this? Instead of bothering with this, using a handy API would be much better.
Generally, there is no API for this, unfortunately.
Believe me, I looked and looked and looked.... for an earlier project.

If there is one in the graphics driver, let me know -- because I want to know too!
fuzun wrote:It is necessary to detect missed frames, but it is not easy I think. Using microsecond clock can only yield predicted results. And if this result is used, it can even break a working strobing frame sequence.
But an incorrect guess doesn't matter!

Missed frames means you are out-of-sync anyway, so you only need to guess correctly 50% of the time for the guessing algorithm to be worth it!

So you only need better than 50% accuracy to make this guessing algorithm worth it.

But I think you can achieve 98% accuracy for freezes up to ~1 second long, in my experience with microsecond timers.

Example: Say, one VSYNC interval is 0.0166945 second (running average time between returns from pageflips for the last 10 seconds, which becomes accurate to sub-millisecond timescales because of averaging).

Now, you had a computer freeze that resulted in a tme of 0.568613 seconds long (this number made intentionally inaccurate to 0.001 second -- 1 milliseconds -- returns from VSYNC pageflip calls are usually successfully more accurate than that whenever you're trying to pageflip full-throttle).

Now, divide 0.568613 by 0.0166945 and you get 34.06 ... That is indeed almost an integer (this number is inaccurate to only 0.06) so trust the number. It's an even number -- 34 refresh cycles elapsed between page flips -- so we can decide to correct the phase flag since it didn't alternate as usual (as it would with odd numbers -- as if only 1 refresh cycle elapses between page flips)

See, I injected a full millisecond of error into this sample math, and I was still able to to correctly guess the phase. Microsecond timers are accurate enough, and one millisecond of error doesn't destroy things enough to fail to guess.

Basically, keep a running average of vsync intervals (Basically time deltas between page flips). That's your refresh interval.
Now a delta becomes more than ~1.9x your vsync time average, you probably have a missed frame. Divide this number by your current vsync time average. If resulting is within 0.1 or 0.05 of an integer, then execute your correct-phase guessing algorithm. Do some common sense processing. More than 1 seconds passed between page flips = do nothing (too risky to guess after that long a freeze). Not integer = likely time inaccuracy, do nothing. Odd integer = do nothing (you're in sync). Even number = 99%+ chance you are definitely out of sync. Change phase!.

So to summarize:
-- Get microsecond timestamps right after return from blocking pageflip call (whatever you use).
-- Get the difference.
-- If more than 1 second elapsed, then ignore below steps. (aka, it's an outlier, don't bother guessing or updating)
-- If it looks like 1 vsync interval, put it as part of the ongoing running rolling average time interval between VSYNC's. (Throw away outlier numbers). This number becomes submillisecond-accurate very very fast, just watch http://www.testufo.com/refreshrate to understand what I mean.
-- If it looks like within ~0.1 of being an integer and it is 2 or more vsync intervals, do this:
......odd number: don't do anything
......even number: swap the internal phase flag (don't have to do actual visible phase swap, just do correction to the phase variable)

So what if it guesses incorrectly sometimes? Big whoop -- at least you're doing better than doing no phase-correction algorithm, because it only needs to guess correctly more than 50% of the time to be an improvement over doing nothing at all! :D

The ongoing phase swaps after incorrect guesses will automatically (within seconds) fix any problems from incorrect guesses. The goal is simply, on average, to do better than doing nothing at all about computer freezes and framedrops.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: Another software strobing implementation on a 3D engine

Post by fuzun » 10 Feb 2018, 16:16

Sorry for the delay. I have been busy recently due to school things.

Fork page: https://github.com/fuzun/xash3d/commits/strobe-support

Strobe implementation v2 is pushed to the repository!

Here is what was done:
  • Updated code base to support burn prevention - swapping algorithm properly. Almost rewritten from scratch.
  • Added a cvar to open debug mode: (r_strobe_debug)
    Image
  • PWM (Simulated) Stats are now shown in the debug mode stats!
    Here is the algorithm:

    Code: Select all

    strobeinterval: r_strobe console variable.
    Frequency: (1 / ((1 / (float)(curfps))*(abs(strobeInterval) + 1)))
    Duty Cycle: ((1 / (float)(abs(strobeInterval)+1)) * 100) * (strobeInterval < 0 ? -strobeInterval : 1)
    
  • Brightness details are now shown in the debug mode stats.

    Code: Select all

    #define calculatePercentage(x, y) \
    	(100 * y / x)
    #define actualBrightnessReduction(fps, efps) \
    	((fps - efps) * 100 / fps)
    #define logBrightnessReduction(base, fps, efps) \
    	actualBrightnessReduction( log(base) , log(base * calculatePercentage(fps,efps) / 100) )
    #define squareBrightnessReduction(base, fps, efps) \
    	actualBrightnessReduction( sqrt(base) , sqrt(base * calculatePercentage(fps,efps) / 100) )
    #define cubicBrightnessReduction(base, fps, efps) \
    	actualBrightnessReduction( cbrt(base) , cbrt(base * calculatePercentage(fps,efps) / 100) ) 
    
  • Fixed strobe issue when windowed & in console.
  • ...
What needs to be done:
  • Making menu entry for enabling strobe mode.
  • Debug stats are indirectly synced with the strobing algorithm. There needs to be direct synchronization.
  • Keeping internal phase on track with monitor's phase using high precision timer. - Thanks Chief!
  • Try to make swapping seamless by rendering non opaque frames when swap function gets triggered. - Thanks Chief!
I looked the simulated PWM stats and I was amazed at the information it gave.

r_strobe 1 -> PWM Frequency is: 72 Hz and PWM Duty Cycle is 50%
r_strobe 2 -> PWM Frequency is: 48 Hz and PWM Duty Cycle is 33.33%
r_strobe 3 -> PWM Frequency is: 36 Hz and PWM Duty Cycle is 25.00%
r_strobe -2 -> PWM Frequency is: 48 Hz and PWM Duty Cycle is 66.66%
r_strobe -3 -> PWM Frequency is: 36 Hz and PWM Duty Cycle is 75.00%
r_strobe -50 -> PWM Frequency is: 2.86 Hz and PWM Duty Cycle is 98.03%

And for brightness;

r_strobe 1
Image

r_strobe 2
Image

Now I understand why r_strobe 1 is the best option. Because the brightness reduction (perception) is not even 50% (Probably near 30% considering 300cd/m^2 brightness on the monitor) and it provides the highest PWM frequency.

Also, brightness stats are pretty much useless below 50 Hz because I think most people will notice flickering at that point. So I will need to do something about it.

My main priority now is to syncing internal and external (monitor) phases and making swapping seamless.

I do not know when or if they will be done in near future because I can not focus on this project due to school things etc. Because of this I think I will implement the menu entry and take a break that will probably last months.

I suggest you to look at the repository and contribute as much as you can.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Another software strobing implementation on a 3D engine

Post by Chief Blur Buster » 12 Feb 2018, 18:39

Fantastic work!

I realize you have to focus on other things, and this is not something you can continue full time on, but we appreciate your contribution to the open source community!

Hopefully other people can utilize variants of this work in other projects (e.g. emulators, etc).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: Another software strobing implementation on a 3D engine

Post by fuzun » 13 Apr 2018, 17:34

I have ported this to C++ (more like C with classes now. It looks much more pretty) and made it stand-alone.

https://github.com/fuzun/strobe-api

I will try to update this repository regularly. I hope it will be used in other projects so that we will be able to play games without motion blur.

It is a quick port and needs a lot of correction, but it works.

Image

The simulation on the console runs on FPS value randomly generated in range (99, 101).

fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: Another software strobing implementation on a 3D engine

Post by fuzun » 27 Jun 2018, 20:51

Hi again,

Its good to see that there is a new forum for software development.

I have made a multi-resolution supported Flappy-Bird clone 2D sandbox game with Qt for trying these kind of stuff: https://github.com/fuzun/Flappy-Bird-Qt .

Image

Today, I integrated StrobeAPI(https://github.com/fuzun/strobe-api) into the game.

StrobeAPI, in its current form is extremely fragile and its code is very dirty (its like C with classes now) so it should not be used in production environment. But in future I will try to improve it.

My fork of Xash3d engine (https://github.com/fuzun/xash3d-strobe) also includes this library but it includes the C version of it and it will not be in sync with the master repository because I will not continue providing C version of the library as it needs to be object oriented and C does not provide that. C version contains a lot of macros to simulate object oriented code.

So I think that this demo game is a proper example for integrating initial version of StrobeAPI into a relatively larger scale C++ project (compared to commandline test.cpp, which is removed now) .

Integration with StrobeAPI is very easy, in this case it is done like this:

Code: Select all

        bool show = strobe->strobe();
        if(show)
            QGraphicsView::paintEvent(event);
        else
        {
            QPainter painter(this->viewport());
            painter.fillRect(event->rect(), Qt::black);
        }
Here is a screenshot when Strobe is enabled:

Image

The game includes a "config.ini" file in which you can set game and strobe parameters:

Code: Select all

[General]
Fullscreen = 0 ; 0: Window mode, 1: Fullscreen mode
; Both OpenGL and software rendering work better in fullscreen mode.

ScreenWidth = 480
ScreenHeight = 800

Sound = 0 ; 0: Sounds are disabled, 1: Sounds are enabled
; Sounds may cause stuttering.

;ScaleFactor = 1.0 ; Use if you are not satisfied with the default value.
; Used for scaling images.


[Graphic]
OpenGL = 1 ; Determines what to use for rendering. 0: Qt Raster-Software (CPU), 1: OpenGL (GPU)
; OpenGL support is experimental. Tests show that raster is better performance-wise.
; Swap interval of raster is 0. It might cause tearing.
; Swap interval of OpenGL is 1. V-Sync is enabled.

Antialiasing = 0
SmoothPixmapTransform = 1
HighQualityAntiAliasing = 0
FPSCap = 1 ; 0: capping disabled, 1: capping enabled
; Only valid in software rendering.
; Viewport update mode determines the dynamically capped fps.

ViewportUpdateMode = 1 ; 0: FullViewportUpdate, 1: MinimalViewportUpdate, 2: SmartViewportUpdate
; Only valid in software rendering.
; Only valid when FPSCap != 0


[Physics]
TickRate = 5 ; in milliseconds
DisableCollisionDetection = 0
;SpeedFactor = 1.0 ; E.g. 2.0 makes the game 2x faster (except bird's movements).
; OpenGL rendering causes slower pace so this parameter may help for setting up the correct pace.

ComplexAnalyse = 1 ; Complex collision detection analysis enabled or disabled.
; Only valid when DisableCollisionDetection = 0


[Strobe]
Enabled = 1 ; StrobeAPI enabled or disabled
; Strobe mode requires OpenGL renderer.

Dialog = 1 ; StrobeAPI information dialog is shown or not.
; Affects Strobe performance in a bad way.
; Needs StrobeAPI to be enabled.

Method = 1 ; Strobing method.
; 1 : [RENDER - BLACK]
; 2 : [RENDER - BLACK - BLACK]
; -2 : [BLACK - BLACK - RENDER]
; 0 disables strobing

CooldownDelay = 3 ; When deviation of fps becomes too high (instability) , strobing gets temporarily disabled for a time period. This parameter sets the time period in seconds.
; Set 0 to disable cooldown.

SwapInterval = 0 ; Phase swap interval in seconds.
; Set 0 to disable phase swapping.
; Setting 0 may cause image retention in some monitors.

DeviationLimit = 5.0 ; Standard deviation limit of collected FPS values in a period. When deviation becomes higher than the limit, strobing will be temporarily disabled.
; Only valid when CooldownDelay != 0

DialogUpdateInterval = 20 ; Update interval of information dialog in milliseconds.
Binaries can be downloaded here: https://github.com/fuzun/Flappy-Bird-Qt ... t-v1.0.zip (Qt binaries are included)

The game runs >1000 FPS when FPS cap is disabled and software rendering is used. And it runs with vertical sync enabled when gpu rendering is used. Strobe needs v-sync so gpu renderer must be used if strobe needs to be activated.

User avatar
RealNC
Site Admin
Posts: 3741
Joined: 24 Dec 2013, 18:32
Contact:

Re: Another software strobing implementation on a 3D engine

Post by RealNC » 28 Jun 2018, 05:29

I just built this on Linux. Only a couple minor code changes were required (will do a github PR later). Seems to work fine, except of course that Linux desktops suck and strobing is not always in sync :mrgreen:

Image
SteamGitHubStack Overflow
The views and opinions expressed in my posts are my own and do not necessarily reflect the official policy or position of Blur Busters.

fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: Another software strobing implementation on a 3D engine

Post by fuzun » 28 Jun 2018, 08:29

RealNC wrote:I just built this on Linux. Only a couple minor code changes were required (will do a github PR later). Seems to work fine, except of course that Linux desktops suck and strobing is not always in sync :mrgreen:

Image
Strobing is not in sync for me, too (I use Windows). I think it is related with Opengl rendering and Qt-Opengl interaction.

When OpenGL rendering is enabled, the game will work in a slower pace and will stutter all the time. Especially when Strobe Dialog is open and the game is windowed.

Maybe graphics driver or operating system do not give priority to an app when the app is not fullscreen ? or when app pops up a dialog ?

In fullscreen / dialog closed mode it works much more smoother, but still not smooth enough for proper strobing.

There is a QTimer with 0 interval that updates the viewport (which invokes paintEvent). I think the problem is this.

I need to find a way to sync CPU and GPU instead of telling CPU that GPU must redraw the viewport whenever CPU is able to tell (interval = 0).

Instead of this QTimer approach, the default FullViewportUpdateMode works too but it is not suitable for strobing because it only redraws screen (paintEvent()) when there is an actual update in the GraphicsView.

Too bad that swap interval of software renderer is 0. I tried but could not set it to 1. So in manual viewport update mode I could not find a way to sync gpu with screen in software rendering mode. It makes sense because it is software renderer after all.

fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: Another software strobing implementation on a 3D engine

Post by fuzun » 04 Jul 2018, 18:43

Strobing in PPSSPP:

Image

See how ease it is to integrate StrobeAPI into a big project: https://github.com/fuzun/ppsspp-strobe/ ... bb126083f8

However there is a (big) problem. Even when I enable vsync, I could not get PPSSPP working in native refresh rate of screen. It is always 60 fps in maximum. It is probably set that way to comply with speed of emulated games. Maybe it is better to insert black frames after rendered frames instead of replacing existing rendered frames to black ones.

1) 60 fps -> 120 fps (60 rendered + 60 black)
2) 60 fps -> 60 fps (30 rendered + 30 black)

2. Method is what is being done now. But 1. Method should be better when refresh rate of screen is not achieved like in this case. It is much harder to implement 1. method and I do not know much about emulators in general so I do not know what to do to not screw up game timings while using this method. In future I may try to implement it but I need to learn PPSSPP internals first.

Post Reply