PCSX2 (and others) with software-based black frame insertion

Talk to software developers and aspiring geeks. Programming tips. Improve motion fluidity. Reduce input lag. Come Present() yourself!
Post Reply
User avatar
masterotaku
Posts: 436
Joined: 20 Dec 2013, 04:01

PCSX2 (and others) with software-based black frame insertion

Post by masterotaku » 10 Sep 2018, 02:02

I have been adding more and more features to my PCSX2 3D Vision fix: https://helixmod.blogspot.com/2018/01/pcsx2-dx11.html

Most of them related to 3D, and some others that can work for everyone. The one I'm going to talk about is black frame insertion.

30fps games in this emulator always show 60fps in frame counters, because the emulator duplicates whatever internal fps the game has to 60fps (50fps for PAL games when you can't play at 60Hz). With my fix, pressing the "n" key turns alternate frames black, using an odd/even frame counter I set up in 3Dmigoto and turning black two types of pixel shaders. As I say in the blog post, the counter doesn't know if a frame is the original or the duplicated (it's random when you start a game), so pressing "n" again will switch to the other frame. Pressing it again will disable black frame insertion.

If you know (and care) how to look for the first frame, you should get the same visual and input lag as not using BFI. If you use the two methods of BFI at 120Hz (more on that after this), the comparison is like this for a 30fps game (F=frame, X=black frame):

No BFI: FFFF
BFI: FXXX or XFXX

Obviously, if you use this BFI feature with a 60fps game you are effetively downgrading that game to 30fps.


There's a second black frame insertion method I made with the "u" key that only 3D Vision users can use. It puts the game in 2D and makes the right eye image black (because the left eye image always comes first). Its purpose is to show 60fps with BFI at 120Hz for people that don't have access to 60Hz strobing, or for people that want to switch between 3D and 2D with a click without losing perfect vsync and low motion blur. This method can be combined with the other one. People with 3D Vision will most likely want to play in 3D, but I wanted to make this extra feature.

Here's video proof:

Watch this one at 60fps, and preferably with your monitor at 60Hz with strobing: https://youtu.be/LrJwGU0grPE
And this one should be viewed at 4x the speed at 120Hz, because it's a slow motion video at 30fps (120fps in real time): https://youtu.be/XD0jEnoS3Bw

I recorded both with my phone, using both BFI methods stacked and getting the first frame. Sorry for the black horizontal bar in the first one. It's because of synchronization with my phone. And I had to use very low monitor brightness to not get a super saturated video (maybe the phone wanted to overcompensate the BFI).

And just yesterday I saw that there's a ReShade shader (https://github.com/BlueSkyDefender/Dept ... rnation.fx) that basically does the same as my "n" hotkey. But mine has the extra preset of using the other frame at least, if you don't get the first frame at first.


My mod requires using the DX11 hardware renderer, and for BFI you should ALWAYS use vsync, and a perfect one at that.

tl;dr: black frame insertion for 30fps games and black frame insertion for 60fps games (3D Vision needed for the last one). Epilepsy warning, and you have to be comfortable with 30Hz strobing.
Last edited by masterotaku on 17 Sep 2018, 12:19, edited 1 time in total.
CPU: Intel Core i7 7700K @ 4.9GHz
GPU: Gainward Phoenix 1080 GLH
RAM: GSkill Ripjaws Z 3866MHz CL19
Motherboard: Gigabyte Gaming M5 Z270
Monitor: Asus PG278QR

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: PCSX2 with black frame insertion

Post by Chief Blur Buster » 11 Sep 2018, 22:38

This is fantastic!

BTW, it's easy to detect which frames are duplicate or not, simply by mathematically totalling all the pixels together. Do this every frame. If the 64-bit integer does not change much, it means few pixels changed (e.g. HUD digits only), this can be used to automatically resync

A variant of this frame-identicalness algorithm back in year 2000 was used along with my 3:2 pulldown deinterlacer in dScaler (Internet Archive) -- I was the inventor of the world's first open source 3:2 pulldown deinterlacer algorithm in the dtV / dScaler application which John Adcock implemented my algorithm into a surprisingly simple Pentium MMX assembly routine which simply created a "frame identicalness" algorithm; it simply totalled all the pixels in the luma channel into one 32-bit or 64-bit integer variable.

Compare the value for adjacent frames. If the difference is zero or extremely small percentage-wise, that means there's been no or almost no change in the frame (e.g. to cover rare things like HUD digits may change at 60fps but the content of frame might change only 30fps, etc). This still worked well with interlaced material before deinterlacing -- metaphorically mathematically totalling odd scanlines and even scanlines, still showed sufficient spike-minor-spike-minor-minor-spike-minor pattern that was easy to threshold on, to determine which fields of 60i material to merge, to create flawless 24p. And re-0framepace it so that it played perfectly at 48Hz, 72Hz and 96Hz. dScaler was the first PC app that could successfully do that; turning DVDs into "24p" in realtime while watching -- a 233Mhz Pentium MMX was able to deinterlace DVDs to 480p/24 in realtime with a Hauppage TV card, using assembly language to process every frame into twenty-four perfect 640x480 framebuffers per second which was then scaled in realtime by the graphics card of the day (e.g. Matrox G200, Riva TNT or GeForc256) and when used with an external DVD Player, actually produced a better picture than Creative Labs PC-DVD Encor Dxr2 Kit which used a lower-quality scaler with no proper deinterlacer. And VHS videotape could become 1080p/24 framepaced perfectly, with a custom refresh rate created by the Entech Taiwan Custom Resolution Utility (the world's only CRU back then). That was almost twenty years ago.

Anyway, just reminiscing on the clever adjacent-frame identicalness detection algorithm (which is relevant here) that was very forgiving of slight differences (e.g. video noise, MPEG2 aritfacts, odd-vs-even, etc) in two identical frames or fields (even if the field-phase was different; odd vs even), while still within the computing envelope of a Pentium 233 MMX CPU to realtime output 480p/24 from a 480i/60 source in the pre-GPU days.

Although today, computers are so much faster, and have lots of memory, you can simply queue a few frames and simply compare the pixels of adjacent frames now. But this was a pretty simple algorithm back then that. People began using HTPCs instead of Faroudja line doublers. It kind of destroyed their dedicated-video-processor market when mainstream graphics & CPUs/GPUs gained enough computing power to do realtime 480p video processing (that milestone occured around year 2000-2001).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
masterotaku
Posts: 436
Joined: 20 Dec 2013, 04:01

Re: PCSX2 with black frame insertion

Post by masterotaku » 12 Sep 2018, 02:20

I'm limited to what 3Dmigoto can do. I don't think I can analyze for example the color of a pixel and save it to then change a variable on the next frame. I would have to ask the 3Dmigoto developers about that or request it. Then even if I can get an average of the pixels or a hash or whatever, then having to somehow resync BFI would be another challenge on its own.

Incidentally I also made my own black frame insertion for RetroArch in DX11 (it looks like RA doesn't have its own BFI in DX11 mode) when I made a conversion from shutter 30Hz/eye 3D games to 3D Vision (https://helixmod.blogspot.com/2018/04/r ... -dx11.html). But I haven't tested it in 2D for more than a minute, so I'm not sure if the two presets I have also can change the shown frame in 2D.

I'll test that today or tomorrow and add the "3D Vision -> 2D" BFI trick too. The main problem I have with RetroArch is a bug caused by the latest Windows 10 update that doesn't let me use fulscreen the normal way. I also want to change how I do my counter so 20fps games can be synchronized in either way correctly in RetroArch. Currently it would only show one out of three frames with the first BFI setting, and the other one would show two out of three frames. And there would be 3 possible frames to use instead of 2... Whew, it looks like I have a lot of fun thinking and testing to do :p.
CPU: Intel Core i7 7700K @ 4.9GHz
GPU: Gainward Phoenix 1080 GLH
RAM: GSkill Ripjaws Z 3866MHz CL19
Motherboard: Gigabyte Gaming M5 Z270
Monitor: Asus PG278QR

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: PCSX2 with black frame insertion

Post by Chief Blur Buster » 14 Sep 2018, 13:33

About Repeat-Frame Detection Algorithm...

For generating a single value (used for frame-identicalness algorithms), the pseudocode is really essentially:

Code: Select all

int total = 0;
for (x = 0; x < framebuffer.width; x++)
    for (y = 0; y < framebuffer.height; y++)
        total += framebuffer[x][y].green;
"total" changes very slightly for identical frames, and "total" changes lots for non-identical frames.
So this could be used as a frame-identicalness detection.

You can just add all pixels, instead of just luma/green. So can just remove ".green" -- it was convenient back then because MMX could add many 8-bit values together simultaneously (rather than the simple pseudocode above), and we piggybacked off that ability to maximize performance by only focussing on the luma/green channels back in the day.

Back then, it was very forgiving of "noise" (differences in MPEG2 artifacts between two identical frames, even-vs-odd fields, video noise, etc) while being very computationally simple.

This made it very good for 3:2 pulldown detection on MPEG2 sources, where one often had to compare the imagery of only even scanlines, versus the imagery of only odd scanlines, to see if the even scanlines and odd scanlines were from the same frame, or from different frames. This super-simple algorithm brilliantly worked for that -- big differences in "total" occured if the even scanlines versus odd scanlines were different frames.

We also noticed that this simple algorithm was apparently was resistant to things like a 60fps (unique-fields-per-sec, obviously, being 480i) scrolling CNN ticker when most of the screen surface was running at 24fps (e.g. CNN showing a 24fps film snippet above a 60fps ticker), the software 3:2 deinterlacer still clearly prioritized on the "dominant framerate" of the material in multiple-framerate material -- despite the deceptive simplicity of this algorithm.

Graphing the threshold spikes/valleys on a graph, you could then define a threshold where one decided two frames (or fields) was identical or were different, despite all that "noise" fighting each other.

That said, interlacing is not important here, and with perfect GPU renders -- you might now get zero-difference between identical frames now (since repeat-frames may produce exactly the same bit data), but a threshold may be useful anyway since some games weirdly have 30fps content with small amounts of 60fps content (e.g. animations at 30fps but something else like HUD is 60fps)

We were able to realtime math this on Intel Pentium MMX assembly to deinterlace 480i/60 into 480p/24 flawlessly and output with smooth framepacing to 72Hz CRTs (including CRT projectors).

_______

About Black Frame Insertion...

Have you read what was done with with fuzun's work on xash3d black frame insertion ... I explained how sometimes software BFI creates burn-in on certain displays and gave him a solution how to solve that burn-in problem.

Question: Does your software BFI have an anti-burn-in compensation algorithm?
It doesn't affect all monitors, but it affects some.

More detail: Software BFI on certain monitors can create accelerated burn-in (image retention) effects that uses a rapid positive-negative-positive-negative LCD inversion algorithm (alternating voltage polarity for pixels in a refresh cycles are common to keep the LCD panel voltage-balanced).

Software BFI appears to inconveniently mucks up with that voltage balancing logic of LCD inversion (normally a checkerboard/lines pattern) and accidentally creating a voltage imbalance when the corresponding pixels of black frames are a specific polarity and their counterpart pixels in nonblack frames are the opposite polarity. After roughly 1+ minutes of this being sustained on certain monitors, this causes image retention (LCD burn-in, essentially) that requires several minutes of video playback to fully erase. Long-term use can potentially create more permanent effects, so anti-burn-in logic for software BFI implementations are now highly recommended.

Few people know this, but since 2013, I modified some TestUFO animations, including the TestUFO flicker test even intentionally inserts a phase-change once every several seconds, for this very reason. I explained how to solve that problem by occasionally switching phases (e.g. extending or shortening a 2-frame-repeat to either 1-frame or 3-frames once every 15 or 30 seconds) and even explained how to reduce the flicker of phase-changes (use an alpha-blended 50%, or perhaps 2.4-gamma-corrected alphablend) so that the 2-2-2-2-2-3-2-2-2-2 cadence-change in BFI does not have a noticeable flicker.

And then fuzun came up with an algorithm that seemed to further make the antiburnin-algorithm more completely invisible by switching phases more/less often depending on how likely burn-in has happened. But you don't have to go that elaborate, just a simple optional command line arg (e.g. --phasechange X) where X is seconds for switching phases in software BFI, typically 15 or 30.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
masterotaku
Posts: 436
Joined: 20 Dec 2013, 04:01

Re: PCSX2 with black frame insertion

Post by masterotaku » 16 Sep 2018, 13:39

Before reading your comment, instead of checking RetroArch again I added BFI to my SEGA Mega Drive & Genesis Classics (https://helixmod.blogspot.com/2018/06/s ... ssics.html). The special thing about this one is that BFI only applies to the TV in the virtual room and the fullscreen emulator image, while the rest of elements are still working at 120fps.

Video showing that in Sonic: https://youtu.be/VeaCrCfcKwM


Now about what you said.

So you're saying that I should switch frames every once in a while to somewhat image retention. I've just tried it with my Sega fix, and it works. I added another frame counter that resets every 600 frames. When that reset happens, I switch the odd/even frame counter that I had in the first place and let the shader know that this current frame is a transitioning one.

It clears image retention when I'm not moving, but to be perfect for that (or almost), it has to happen every few seconds (like 4 or 5) in my Asus PG278QR. And the bad thing about it is that the change in timing creates a noticeable flicker at that moment. I've tried to minimize it by reducing brightness in the extra non strobing frame, but it's still noticeable even if it's better. There's also the variable visual/input lag problem this causes.


By the way, there's something else that I can do (and did just now), which is doing "incomplete" BFI. That is, instead of black frames, reduced brightness frames. I guess the result is similar to the "240Hz" Eizo Foris FG2421, which does two strobes per Hz at 120Hz, but one of them is small. With for example 10% brightness frames, the visual signs of image retention are greatly reduced, at the cost of a slight "ghosting" in movement (caused by the frame that has reduced brightness). Perhaps I can offer an additional option for this and call it "black frame insertion blending".

None of these extra things are in the fix I published yet. Just the basic black frame insertion presets. The blending one seems good enough to put it there soon with multiple presets or a more granular hold hotkey, but I'm still in doubt about the automatic phase switching one.
CPU: Intel Core i7 7700K @ 4.9GHz
GPU: Gainward Phoenix 1080 GLH
RAM: GSkill Ripjaws Z 3866MHz CL19
Motherboard: Gigabyte Gaming M5 Z270
Monitor: Asus PG278QR

User avatar
masterotaku
Posts: 436
Joined: 20 Dec 2013, 04:01

Re: PCSX2 (and others) with black frame insertion

Post by masterotaku » 17 Sep 2018, 12:28

I have updated the Sega fix with 3 more hotkeys that control the black frame insertion blending. One of them with 10% steps from 0% to 100% (first press starting at 10%), and two "hold" type hotkeys for better fine tuning (+-0.25% per frame held).

Values over 30% are pretty useless, but options are good, I guess. Any float value between 0 and 1 can be set in "d3dx.ini" at line 119, where it says "z4=0", and you can change the "h" hotkey presets at line 321, where it says "z4 = 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1, 0".

I guess I'll do the same for PCSX2 later.
CPU: Intel Core i7 7700K @ 4.9GHz
GPU: Gainward Phoenix 1080 GLH
RAM: GSkill Ripjaws Z 3866MHz CL19
Motherboard: Gigabyte Gaming M5 Z270
Monitor: Asus PG278QR

evanapage
Posts: 1
Joined: 29 Sep 2018, 16:12

Re: PCSX2 (and others) with black frame insertion

Post by evanapage » 29 Sep 2018, 16:21

I get some useful ideas about the PCSX2 with black frame insertion. this is very interesting for me. I have faced Windows 10 Explorer Keeps Crashing this. after reading your information properly I will definitely get benefited.

fuzun
Posts: 17
Joined: 20 Jan 2018, 21:18

Re: PCSX2 (and others) with black frame insertion

Post by fuzun » 30 Sep 2018, 18:03

Hey, I just came up to this thread while browsing the forum.

Though I have yet to find a solution for the occasional flicker happening while phase switchs (I tried lowering brightness too but it does not work any good in my opinion), you might want to look at my generalized work for the black frame insertion thing: https://github.com/fuzun/strobe-api . But note that in its core, it does not do anything more than any other black frame insertion implementations. It just offers bunch of other things in a more standardized way I guess.

Recently I have made it headeronly library so that integration will be a lot easier. In my opinion, it now can be used in production.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

PCSX2 (and others) with software-based black frame insertion

Post by Chief Blur Buster » 02 Oct 2018, 15:26

I've thought of something new for improving phase-switch-flicker-supression of anti-burnin algorithms of software-BFI:

Occasional flicker during phase switches:
Try 2.2-gamma-correcting the phase mode switches, instead of 50% alphablend-darken.
That said, overdrive imperections may still have some remnant faint flicker during phase switches.

Basically, phase-mode-switch flicker can be made virtually invisible, by using a 2.2-gamma-corrected frame to 50% brightness per pixel (physical brightness of each pixel, rather than RGB brightness). Then phase switches won't flicker on well-overdriven high-Hz monitor. A 50% alphablend (halving RGB values for a phase-mode-flicker-supression algorithm) is not a perfect halving of the number of photons hitting the eyeballs due to the gamma correction monitors do.

I use gamma correction during http://www.testufo.com/gtg-vs-mprt to try to match the greyshades of the bars.

I believe that fuzun made an algorithm that automatically decides to delay phase switches for imagery that is dark (because BFI on dark/dim imagery is less likely to burn in quickly as BFI on bright imagery). At least, that's my understanding, and he even accomodates weird BFI algorithms like 180Hz for 2:1:2:1:2:1 BFI cadences in either 66%:33% or 33%:66% ratios. (Also supports 4-cycle BFI in 25:75%, 50%:50% and 75%:25% ratios) with automatically deferred or disabled phase switches depending on the cadence, since odd-count-cycle BFI (e.g. 3-cycle) such as 60fps@180Hz using 2:1 or 1:2 cadence doesn't need anti-burn-in compensation for software BFI.

Some monitors are fine with 60 or 300 seconds between phase switches, while others work better with 10 or 15 seconds. I just simply used 10 or 15 seconds fixed intervals with testufo.com/flicker

So anti-burnin-compensation (for accidental defeats (caused by software BFI) of LCD inversion voltage balancing) can be as simple as just inserting an extra black refresh cycle once every 10 seconds. Or as elaborate as what fuzun has done with xash3d open source code.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

Post Reply