[Mouse Mod HOWTO] Input lag test setup for Rocket League

Everything about latency. Tips, testing methods, mouse lag, display lag, game engine lag, network lag, whole input lag chain, VSYNC OFF vs VSYNC ON, and more! Input Lag Articles on Blur Busters.
HalfwayDead
Posts: 34
Joined: 18 Jul 2017, 07:27

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by HalfwayDead » 01 Aug 2017, 17:14

Chief Blur Buster wrote:They're probably measuring screen centre, VSYNC ON.
They're definitely not measuring screen centre. They say the signal processing at 144 Hz is 10.2 ms and at 60 Hz it's 7.4 ms and they have monitors reviewed with times as low as 2ms which would be impossible at the centre of the screen with VSYNC ON. Funnily enough they say how they obtained their data for everything in their test except the signal processing. I might write a mail and ask them.

As for my other problems:

I switched to a green LED which is way brighter with the same resistor even though it's supposed to require almost twice the voltage of the red one. Yes, I know the eyes are more sensitive to green light but so are camera sensors with a bayer filter. I also moved it right in front of the lens so it's very obvious now when it's on and when not. It's also easy to see the rolling shutter this way since the LED is sometimes turning on halfway into the frame. In fact it works so well it should be possible to get sub millisecond accuracy of the LED turn on time just by looking at where the first bit of green shows but it obviously doesn't work the same way for the screen. And I should be able to figure out how long it takes for the LED to turn on.

Smashing my mouse button hard didn't seem to make any difference for how fast the LED turned on. But I had a different idea. I can just use the button on the breadboard which works fine. Essentially doing the opposite of the setup you proposed. Not pressing the mouse button to also light up the LED but click the other button to also trigger a mouse click. Do you guys think there is any reason I shouldn't do this? I know that it is not a perfect representation of how the mouse will perform but I can't see a brand new gaming mouse button being any worse than one of those cheap buttons that seem to work fine.

Anyway, I repeated my test with the new setup (50 trials again). Same min/max. Median dropped to 14 ms and mean dropped by 0.66 ms. Considering that the LED now covers the entire side of the camera which means that I can see it even if turns on at the latest possible moment during the rolling shutter scanout, you'd expect the times to go up. So I think it's pretty safe to say that the actual mouse button was causing extra delay.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by Chief Blur Buster » 01 Aug 2017, 18:06

HalfwayDead wrote:They're probably measuring screen centre, VSYNC ON.
Excuse me, they may have changed or added to their test methods, but DisplayLag.com says this:
DisplayLag.com wrote:As of June 24th, 2013, Display Lag, in accordance with sites like CNET, HDTVTest, Sound+Vision, and Anandtech, has shifted to using averages as the standard of grading using the Leo Bodnar Lag Tester, instead of the bottom measurement. As a result, some information in this article is outdated. You can visit the Testing Method page for more information. The original text of this article is preserved for reference.
Their Testing Method Page seems to have disappeared, but you can see this on Internet Archive which says:
DisplayLag.com wrote:The tester presents 3 flashing bars on the top, middle, and bottom of the screen. [/b]Displays marked with “AVG” are calculated using the average of all 3 bars[/b]. Displays marked with “BTM” are calculated using the bottom flashing bar (typically the area with the most lag). As of June 24th, 2013, all newer measurements will be reported as “AVG” on Display Lag, as it is the grading standard agreed upon between Display Lag, CNET, Sound+Vision, HDTVTest, and Anandtech. Displays marked with “BTM” measurements can be updated to “AVG” measurements at any time without notice.
Most of the time (majority of displays) average lag is practically equal to centre square. It can vary (e.g. scanline-optimized overdrive algorithms, like those commonly used with LightBoost, or on displays with very bad/mis-adjusted strobe crosstalk like XL2720Z Version 1 firmwares).

While it is possible they may have changed their testing method since, this is what they have said.

Regardless, I'm sure we can agree that comparing input lag numbers is fiendishly difficult. One tester might have measured to beginning of LCD GtG. A different tester might have measured to the 50% midpoint of GtG (e.g. exact middle grey during a Black-to-White transition).

This is greatly complicated by the fact, that even on a slow 10ms LCD, the first 1-2ms may still begin to emit human-visible photons already (as a pixel gradually fades from one color to the next), and the reaction-time clock (~150ms human reaction time) might begin early in a LCD GtG cycle rather than late.

We know that VA panels might have 5x slower pixel response for certain colors (e.g. Black->DimGray or DimGray->Black) than for other colors (e.g. DimGray->LightGray or Black->LightGray etc.) So your reaction time will be much slower in dungeon video games, than during above-ground games.

This LCD human-reaction-time GtG error margin is almost gone when it comes to 1ms GtG TN panels, since their overdrive has been well-optimized. Almost all of the "1ms TN panels" now emit a human-reactable amount of photons in the first 1ms of any pixel transitions between any possible color pair.

Not all testers tell you how they factor in GtG. Our lag testing standard will be to measure to the 50% GtG midpoint since it's a very accurate compromise.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

HalfwayDead
Posts: 34
Joined: 18 Jul 2017, 07:27

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by HalfwayDead » 01 Aug 2017, 18:12

Oh, now I see why you thought that. I was never talking about displaylag.com. I said prad.de which is a german website with very extensive monitor reviews similar to tftcentral.co.uk.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by Chief Blur Buster » 01 Aug 2017, 18:14

HalfwayDead wrote:Oh, now I see why you thought that. I was never talking about displaylag.com. I said prad.de which is a german website with very extensive monitor reviews similar to tftcentral.co.uk.
(Apologies for confusion)

Yes.

As you see....different testing methodologies for different websites!

You can see how difficult it is to compare numbers between different input/display lag testing methodologies. :(

The best you can do is to publish/document your lag testing methodology. Are you focussing on display lag? Mouse lag? Signal processing lag? Combined display lag (signal + panel)? You use high speed video? You use SMTT? Which camera you used? (shutter accuracy differences between cameras)? You use Leo Bodnar? Screen Top/Center/Bottom/Average/FirstReaction? Or all numbers? You use an oscilloscope with connected to cable VSYNC signal + onscreen photodiode? Etc, etc, etc. And on the method, do you measure to first faint visibility (early GtG) or 50% visibility (GtG midpoint) or full GtG? Etc. Etc.

At the end of the day, input lag is incredibly difficult to standardize input lag measurements because of large numbers of variables. That makes it incompatible to compare numbers between different websites. Even a small refresh rate change and a settings change (strobed vs unstrobed vs VSYNC ON vs VSYNC OFF) can change input lag numbers.

Whether it be GtG, even GtG differences between colors (On VA panels, more display lag in darker dungeon games than above-ground games), or signal processing, or the panel, the scanout velocity (Large Vertical Totals accelerates top-to-bottom scan velocity), whether the display is global flash or top-to-bottom scanout, etc. I've even seen a corner of a VA screen having a little bit less input lag, because it was very hot (power supply behind the screen) -- cold LCDs have slower pixel transitions (you know, forgetting a phone in a freezing car in the middle of a Canadian winter) which can create lag differentials if the panel is an inconsistent temperature throughout! (In the case of dark dungeon games on certain panel technologies like VA panels that has huge problems with making specific dark colors fast, relevant)

Anyway, input lag number differences between different testing methodologies is an incredibly complex universe.

Just publicly document your methodology, as detailed as possible, and be golden. :)
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

Sparky
Posts: 682
Joined: 15 Jan 2014, 02:29

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by Sparky » 01 Aug 2017, 18:14

Do you guys think there is any reason I shouldn't do this? I know that it is not a perfect representation of how the mouse will perform but I can't see a brand new gaming mouse button being any worse than one of those cheap buttons that seem to work fine.
There will be different bounce characteristics, which can change the amount of time between the LED lighting and the mouse registering a press. As long as you consistently use the same switch it shouldn't be a problem. Mainly it's just a question of what you're trying to measure.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by Chief Blur Buster » 01 Aug 2017, 18:26

Sparky wrote:There will be different bounce characteristics, which can change the amount of time between the LED lighting and the mouse registering a press. As long as you consistently use the same switch it shouldn't be a problem. Mainly it's just a question of what you're trying to measure.
And a new switch.

The leading-edge trigger of a fresh new button (after the mouse button hasn't been touched for a moment -- expiring any antibounce processing of previous mouse button presses) -- produces a strong pulse, so quickly, that good proper antibounce processing should not have to insert any meaningful delay.

I'm pretty sure there's a lot of crappy mouse that adds lots of delay -- but for the real eSports outfits that actually focuses on optimizing their mouse firmwares to avoid lag, it's possible to write antibounce code that gives a lag-free-pass to the leading edge of a button press event (as much as the performance of the mouse motherboard allows -- aka processing overhead in the mouse microcontroller). Ideally, that is how it should be -- leading edge of a button press of a fresh mouse button switch, should be essentially lagless (at the millisecond timescales). At least, not measurable difference between the LED and the mouse electronics deciding it's a mousedown event. At the very least, the LED is also measuring mouse performance too -- you're measuring buttons-to-pixels anyway -- so button lag and (any unneded leading-edge debounce lag) -- essentially becomes part of the buttons-to-pixels equation. Document the full system you're measuring input lag on (from mouse all the way to the display, and the computer/GPU in between.)

The resistance change from a button press of a good, brand new mouse button switch is so fast and so dramatic (microseconds, literally) that it should get a free pass at the leading edge right there and then. Antibounce would simply need to kick in to avoid premature mouseup event (after mousedown) -- leading bouncing. Or avoid premature mousedown event right after a mouseup -- trailing bouncing.

But the leading edge of the first mousedown, if a full-strength signal (darn near squarewave resistance change), the button can be immediately flagged right away as a mousedown event. Ideally, no antibounce delay *should* be needed for leading mousedown edge if the mouse manufacturer has written their antibounce code properly. Many great people worked on mouse button lag on OCN...

Certainly, if the resistance change is more gradual (more worn switch), it will have to wait until it's a really clear signal.

On some mice, the antibounce logic is adjustable/tweakable (e.g. le (which would produce more erraticness with worn mice buttons). Just putting slight pressure on a mouse button without clicking it, can create false-positive signals, but that is also usually a situation of a not-quote-infinite resistance change to not-quite-zero resistance.

Regardless, for good brand/recent mice, fresh button, it's far below the noisefloor of current cheap highspeed cameras (e.g. 1ms granularity of 1000fps cameras).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

HalfwayDead
Posts: 34
Joined: 18 Jul 2017, 07:27

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by HalfwayDead » 01 Aug 2017, 19:07

Yeah it's definitely hard to compare.

I will keep testing with the external button for now as I just want to get a baseline for my system. Different framerates/refresh rates/VSync etc. Like I said in my first post, I want to compare different controllers since they are mainly used for Rocket League and compare them to what should be the fastest: a 1000 Hz mouse. I will definitely have to use the internal buttons then because otherwise it wouldn't really be fair. Which in return also kind of means that I can't compare some of my used controllers to new ones :(

HalfwayDead
Posts: 34
Joined: 18 Jul 2017, 07:27

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by HalfwayDead » 02 Aug 2017, 16:45

So I did some CS:GO testing to compare my results. Everything in the setup exactly the same although the FPS counter did show 1024 for some reason. Multicore stuff off. Over 50 trials I found 15.1 ms vs. 14.2 ms in Rocket League.
But take this result with a grain of salt. I used the black and white CS:GO map and bound +left to the left mouse button but the speed of how fast the camera turns seems to be unchangeable because they removed cl_yawspeed and m_yaw doesn't change it or do you guys know a different way? Well because of the rather slow turning speed (at 1000FPS) it's really hard to see if there are actual changes or if I'm just seeing noise. Either 1 of three things happened with the results.
  1. There are changes so small that are impossible to see with the camera causing my results to be too high
  2. My results are just as accurate as the ones from RL
  3. My results are too low because every time I wasn't sure whether it was noise or actual change, I listed it as change
I'm probably gonna make a new topic for when I collect more data since this one is about how to set up an input lag test. I just found it interesting that RL seems to be at least as fast as CS:GO which is often used as the gold standard for low input lag. Rocket League runs on UE3 btw.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by Chief Blur Buster » 03 Aug 2017, 11:11

HalfwayDead wrote:So I did some CS:GO testing to compare my results. Everything in the setup exactly the same although the FPS counter did show 1024 for some reason. Multicore stuff off. Over 50 trials I found 15.1 ms vs. 14.2 ms in Rocket League.
But take this result with a grain of salt. I used the black and white CS:GO map and bound +left to the left mouse button but the speed of how fast the camera turns seems to be unchangeable because they removed cl_yawspeed and m_yaw doesn't change it or do you guys know a different way? Well because of the rather slow turning speed (at 1000FPS) it's really hard to see if there are actual changes or if I'm just seeing noise. Either 1 of three things happened with the results.
  1. There are changes so small that are impossible to see with the camera causing my results to be too high
  2. My results are just as accurate as the ones from RL
  3. My results are too low because every time I wasn't sure whether it was noise or actual change, I listed it as change
I'm probably gonna make a new topic for when I collect more data since this one is about how to set up an input lag test. I just found it interesting that RL seems to be at least as fast as CS:GO which is often used as the gold standard for low input lag. Rocket League runs on UE3 btw.
Input lag results will jitter up/down quite a lot (several milliseconds). You can see that from Low/Average/High bars in our charts.

So you will need to run several times (e.g. 10 or 20 runs) and do averages. That's what we did, we counted frames for 5080 pairs of LED-versus-action, for our 240hz article -- it was a ton of work. But our test was unusually extensive... That said, you need at least 10 runs for a single test.

Jitter will become bigger for some game engines (tens of milliseconds), and smaller for others (a few milliseconds). Slower systems (HDD accesses, CPU limits, background software) will jitter in lag more too.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

HalfwayDead
Posts: 34
Joined: 18 Jul 2017, 07:27

Re: [Mouse Mod HOWTO] Input lag test setup for Rocket League

Post by HalfwayDead » 14 Aug 2017, 14:36

I'm really busy with my bachelor's thesis so I haven't been able to work on this much. I'll have more time again in 1 1/2 months.
Chief Blur Buster wrote: So you will need to run several times (e.g. 10 or 20 runs) and do averages. That's what we did, we counted frames for 5080 pairs of LED-versus-action, for our 240hz article -- it was a ton of work. But our test was unusually extensive... That said, you need at least 10 runs for a single test.
I'm not quite sure what you define as a run. If you mean a single test result then I did 50 every test. If you were talking about fully restarting the PC and setting up everything again just in case that it creates variance then I have repeated my baseline test 2 additional times so far. Each of the 50 data point tests had the same average of 14.2ms. I'm guessing that this was a bit of a lucky run and the variance is probably higher than 0.1ms but if the variance is as high as a few milliseconds then this was an extremely unlikely result.

I tried calculating the standard error of the mean but then I remembered that it's not a Gaussian distribution and I never had statistics at school/university and I have no idea how to do it with this weird "custom" probability distribution which would also be different at different FPS/polling rates/monitor refreshrates.

Post Reply