DIY input lag tester.

Everything about latency. Tips, testing methods, mouse lag, display lag, game engine lag, network lag, whole input lag chain, VSYNC OFF vs VSYNC ON, and more! Input Lag Articles on Blur Busters.
Piwi
Posts: 5
Joined: 03 May 2020, 13:32

DIY input lag tester.

Post by Piwi » 23 Dec 2020, 14:26

Hello. I've been trying to make something that would allow me to simply and properly test the input lag of my computer, and I have something I'm pretty happy with. I suppose I'm now looking for validation, and help on whatever I may be doing wrong ?
I plan on writing a guide on what to buy, and how to wire it up, but before spending that time, let's make sure it works as intended.

The tester

This was largely based on that topic in the forums : https://forums.blurbusters.com/viewtopic.php?t=1381, and especially this post towards the end : https://forums.blurbusters.com/viewtopi ... 420#p27989 (many thanks to everyone who participated on that
I've then added some stuff to make life easier for me :

- switched the photodiode sensor from digital to analog, which lets me test on different stuff without having to adjust a potentiometer.

- copied some hacks (can't post external links yet, will edit later, should be first google result if you search "arduino faster analog read") to make analog read on the arduino faster

- added displays on the arduino, to not have to start com port monitoring every time, which was a bit annoying.

- also 3D printed a case, for fun and convenience again.

This is what it currently looks like :
current tester.png
current tester.png (562.7 KiB) Viewed 7703 times
The code

The whole code can be accessed here : (can't post outside links yet, will edit this later :D), but the main part, and what is interesting for us in this post is mainly this part :

Code: Select all

while (digitalRead(hold) == LOW && digitalRead(reset) == HIGH) 
  {
    	float photodiode_state = analogRead(photodiode);
    	float photodiode_low_barrier = (photodiode_state*0.9);
   	float photodiode_high_barrier = (photodiode_state*1.1);
    
   	Mouse.click();
    	startTime = micros();
    	delayMicroseconds(1);
    
    	while( photodiode_low_barrier < photodiode_state && photodiode_state < photodiode_high_barrier && digitalRead(hold) == LOW)
    	{
     	 photodiode_state = analogRead(photodiode);
    	}    
    	endTime = micros();
   	responseTime = (endTime - startTime)/1000.0;
   	
   	delay(random(400, 600));
  }
 
- The delay is there because it sometimes behaves weirdly without it.
- It takes the current light level, then sends a mouse click, and check until it detects a 10% difference in light level.
- I've also done the barriers maths outside of the comparator loop in an effort to make that while run as fast as possible, but I'm not sure how much it matters, if I'm being honest.

Validation

Precision
This is what I did to try and validate it, and make sure it works well enough.

First, my goal was to find out how fast the arduino is without anything in, just the pure delay of the hardware and code.
The test setup for that looked like this :
bare test.png
bare test.png (497.52 KiB) Viewed 7703 times
The whole PC was replaced with a simple LED, and I was
- turning on the LED
- starting timer
- checking the photodiode for light change
- stopping the timer

This gave me results between 30 and 40 µs, over more than 1000 samples. The input lag of a (good) PC being between 10 and 20 ms, this is 0.2% of that. I'd call it good enough.

Consistency
Then, my goal was to see how consistent it is on a real test, with a computer. So I booted up CS:GO, which is (I think) a really consistent game for that, and I started testing.
I averaged my input lag over 500 samples, which should give me a fairly accurate average, and repeated my test 10 times, without changing anything. Results were as follow :
- 14,03748
- 14,05702
- 14,17843
- 14,08252
- 14,0019
- 13,94938
- 14,00252
- 14,01164
- 14,11475
- 14,04328
Which gives me a standard deviation of 0.065, and margins of error that looks like this :
confidence level.png
confidence level.png (10.35 KiB) Viewed 7703 times
Based on this, I think I can say this is reasonably accurate, and as long as tests are done properly, and with enough samples, I think this should be a good way to measure input lag.

Now, I'd like your input, especially if I'm wrong somewhere, or if you think I could improve something.

Thanks !
Last edited by Piwi on 23 Dec 2020, 17:29, edited 1 time in total.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: DIY input lag tester.

Post by Chief Blur Buster » 23 Dec 2020, 17:22

Great stuff! We have developed something like this internally already for our needs (e.g. Blur Busters Approved), but this is a great DIY.

Don't forget latency gradients, as not all pixels refresh at the same time on the screen, www.blurbusters.com/scanout

Also humans can see pixels beginning around GtG10%-ish or so, some lag testers use GtG2% and others use GtG50% to stop the lagtest stopwatch. Don't forget to disclose where you start your latency stopwatch (VBI start? Or Present() start?), don't forget to disclose the monitor/sync technology you use, and don't forget to disclose where you stop your latency stopwatch (Which GtG%? Synthetic or more representative of human vision?)

Retest your tester at 3 positions on the screen, top edge versus center versus bottom edge.

VSYNC ON + nonstrobed creates TOP < CENTER < BOTTOM
VSYNC ON + strobed creates TOP == CENTER == BOTTOM
VSYNC OFF + nonstrobed creates TOP == CENTER == BOTTOM (need 100+ runs to average)
VSYNC OFF + strobed creates TOP > CENTER > BOTTOM (need 100+ runs to average)

For good VSYNC OFF averages, don't forget to randomize position of tearlines via random intervals between measurement reads. This creates your VSYNC OFF waterfall (ranges of input lags).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: DIY input lag tester.

Post by Chief Blur Buster » 23 Dec 2020, 17:39

Piwi wrote:
23 Dec 2020, 14:26
Hello.
Some useful information for DIY lag tester creators:

The above assumes that the panel scans out at the same velocity as cable scanout.

Pixels are transmitted one row at a time over a cable, and most monitors refresh one pixel row at a time. Modern low-lag monitors essentially realtime "streams" those pixel rows onto the screens.

Now, some panels have very interesting scan-conversion behaviours, When that happens, the above rules of thumbs no longer applies, since for VSYNC OFF nonstrobed, I've seen TOP > CENTER > BOTTOM for a reverse-scanning panel, and TOP < CENTER < BOTTOM for a scan-converting panel.

These scan-converting sheninigians are found in Quick Frame Transport thread, most particularly this post.

(P.S. After you have a few separate posts, the forum will let you reply instantly without a moderator queue. It's an antispam measure for new forum members, but you're already posting pretty interesting content very useful to Blur Busters. You may post links with spaces between the periods and colons in http, and I will linkify them for you.)
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

howiec
Posts: 183
Joined: 17 Jun 2014, 15:36

Re: DIY input lag tester.

Post by howiec » 23 Dec 2020, 19:22

Piwi wrote:
23 Dec 2020, 14:26
- It takes the current light level, then sends a mouse click, and check until it detects a 10% difference in light level.

...

Which gives me a standard deviation of 0.065, and margins of error that looks like this : confidence level.png

Based on this, I think I can say this is reasonably accurate, and as long as tests are done properly, and with enough samples, I think this should be a good way to measure input lag.
Nice!

Interesting results and without knowing all the details, it seems pretty accurate and consistent at first glance.

I'm unfamiliar with the hardware and somewhat with the topic in general so forgive my ignorance, however, how did you arrive at the 10% "light level" difference threshold and what photodiode/detector are you using?

Piwi
Posts: 5
Joined: 03 May 2020, 13:32

Re: DIY input lag tester.

Post by Piwi » 23 Dec 2020, 20:32

Chief Blur Buster wrote:
23 Dec 2020, 17:22
Don't forget latency gradients, as not all pixels refresh at the same time on the screen, www.blurbusters.com/scanout
This I knew. As long as the position of the photodiode on the monitor is the same, it should be okay. Especially since I'm mostly doing game tests, and those involve placing the sensor over the muzzle where it flashes, so it's pretty much always on the same spot (or Fortnite, which has a special square that lights up on a side, but it's a fixed spot too)
Chief Blur Buster wrote:
23 Dec 2020, 17:22
Also humans can see pixels beginning around GtG10%-ish or so, some lag testers use GtG2% and others use GtG50% to stop the lagtest stopwatch. Don't forget to disclose where you start your latency stopwatch (VBI start? Or Present() start?), don't forget to disclose the monitor/sync technology you use, and don't forget to disclose where you stop your latency stopwatch (Which GtG%? Synthetic or more representative of human vision?)
I'm actually not sure what I test on the VBI start or OnPresent() start. The arduino acts as a USB mouse, and I do send mouse click -> start timer. I don't know how it relates to those.
Regarding the latency stop point, I don't really use G2G either, since it's games, it's more like grey to white, or grey to yellow or something. I have it as 10% light level change from my sensor, which is faster than I can see it on my monitor (if I put them side by side, I see the arduino reacting slightly before I can see the dispaly changing), so I wanna say it's more synthetic.
Chief Blur Buster wrote:
23 Dec 2020, 17:22
Retest your tester at 3 positions on the screen, top edge versus center versus bottom edge.

VSYNC ON + nonstrobed creates TOP < CENTER < BOTTOM
VSYNC ON + strobed creates TOP == CENTER == BOTTOM
VSYNC OFF + nonstrobed creates TOP == CENTER == BOTTOM (need 100+ runs to average)
VSYNC OFF + strobed creates TOP > CENTER > BOTTOM (need 100+ runs to average)
Those sound fun to test. My monitor (a lenovo Y27Q-20, which uses LG's nano IPS) does not have display strobing, so I'll skip this test, but my behavior seems consistent with what you're saying :
VSYNC ON + nonstrobed -> TOP : 38.25ms | CENTER : 40.24 ms | BOTTOM : 45.60 ms
VSYNC OFF + nonstrobed -> TOP : 20.05 ms | CENTER : 19.81 ms | BOTTOM : 20.54ms (These were done on only 100 samples as it's quite late, but I suspect they would tighten up as I add more values (done on CS:GO at ~600 FPS))
Chief Blur Buster wrote:
23 Dec 2020, 17:22
For good VSYNC OFF averages, don't forget to randomize position of tearlines via random intervals between measurement reads. This creates your VSYNC OFF waterfall (ranges of input lags).
Yeah, random delay between samples is there, however, I don't take waterfalls graphs right now, just average, min and max. I'd need to add outputting values to a PC to do that (which is easily done, but is not in the current version, as I wanted a self contained no software solution).

Piwi
Posts: 5
Joined: 03 May 2020, 13:32

Re: DIY input lag tester.

Post by Piwi » 23 Dec 2020, 20:41

howiec wrote:
23 Dec 2020, 19:22
Nice!

Interesting results and without knowing all the details, it seems pretty accurate and consistent at first glance.

I'm unfamiliar with the hardware and somewhat with the topic in general so forgive my ignorance, however, how did you arrive at the 10% "light level" difference threshold and what photodiode/detector are you using?
Right so, I plan to give detailed hardware and setup in anotehr post at some point, but it's fairly simple :
- an Arduino (needs to be able to be natively recognized as a USB mouse, so Arduino Leonard, or Arduino pro Micro)
- for photodiodes, there are plenty you can find online, for instance https://amazon .fr/dp/B07PJH4P8X/ or https://amazon .fr/dp/B07ZYXHF3C/ (I use the latter, but juste because they're smaller). The only thing you need to care about is that it has an analog output, as I've seen some which are digital ouput only.

For the 10% change, honestly, I kinda picked it because why not.
I first started at 1%, but this gave false positives fairly often, on games where the ambient light was slightly changing at random times.
Then I did 5%, which worked fine, but I wanted to add an extra buffer just in case, so I picked 10%. The input lag measured between 5% et and 10% was also very very close, less than .5ms differences, so I felt 10% was fine.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: DIY input lag tester.

Post by Chief Blur Buster » 23 Dec 2020, 21:18

Piwi wrote:
23 Dec 2020, 20:32
I'm actually not sure what I test on the VBI start or OnPresent() start. The arduino acts as a USB mouse, and I do send mouse click -> start timer. I don't know how it relates to those.
Ok, in your case stopwatch start is the mouse signal. In this case you're doing a fairly full button-to-photons latency test, rather than a GPU-to-display latency test.

So, your latency disclosure is:

Lag Stopwatch Start = mouse event
Lag Stopwatch End = pixel's GtG 10%
Piwi wrote:
23 Dec 2020, 20:32
I have it as 10% light level change from my sensor, which is faster than I can see it on my monitor (if I put them side by side, I see the arduino reacting slightly before I can see the dispaly changing), so I wanna say it's more synthetic.
If you have a 960fps or 1000fps camera you can try to verify this. For the purposes of full chain latency tests, 10% is an adequate trigger, though ideally if you do it in a lookbehind basis, e.g. measure the current color at the mouse trigger event, then measure what color it peaks brightness at (the muzzle flash, etc), and the lag is the 10% point in this color journey. This does require data recording internally in the Arduino and then scanning backwards to the GtG 10% though -- something that might not be easily doable.

Either way, 10% is a good compromise (GtG 10% is even better, since it's the start point of the VESA GtG, and lines up pretty well with human perceptibility, although GtG sub-10% is still visible)
Piwi wrote:
23 Dec 2020, 20:32
VSYNC ON + nonstrobed -> TOP : 38.25ms | CENTER : 40.24 ms | BOTTOM : 45.60 ms
VSYNC OFF + nonstrobed -> TOP : 20.05 ms | CENTER : 19.81 ms | BOTTOM : 20.54ms
(These were done on only 100 samples as it's quite late, but I suspect they would tighten up as I add more values (done on CS:GO at ~600 FPS))
Looks normal, working as designed.
Piwi wrote:
23 Dec 2020, 20:32
Yeah, random delay between samples is there, however, I don't take waterfalls graphs right now, just average, min and max.
This makes sense for simple full-chain latency tests.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

howiec
Posts: 183
Joined: 17 Jun 2014, 15:36

Re: DIY input lag tester.

Post by howiec » 23 Dec 2020, 21:48

Piwi wrote:
23 Dec 2020, 20:41
Right so, I plan to give detailed hardware and setup in anotehr post at some point, but it's fairly simple :
- an Arduino (needs to be able to be natively recognized as a USB mouse, so Arduino Leonard, or Arduino pro Micro)
- for photodiodes, there are plenty you can find online, for instance https://amazon .fr/dp/B07PJH4P8X/ or https://amazon .fr/dp/B07ZYXHF3C/ (I use the latter, but juste because they're smaller). The only thing you need to care about is that it has an analog output, as I've seen some which are digital ouput only.

For the 10% change, honestly, I kinda picked it because why not.
I first started at 1%, but this gave false positives fairly often, on games where the ambient light was slightly changing at random times.
Then I did 5%, which worked fine, but I wanted to add an extra buffer just in case, so I picked 10%. The input lag measured between 5% et and 10% was also very very close, less than .5ms differences, so I felt 10% was fine.
I was just curious which photodiode so that I could look up its datasheet. Just like with any electrical component there are many parameters that can make a difference depending on the application. But I'm guessing their response times are typically in the ns or low us range anyway which is fine for this.

Yeah, I was also curious about your mounting method to minimize any ambient light effects. Clearly, some sort of shroud would help in this regard.

I see, good that the 10% threshold isn't completely arbitrary.

Piwi
Posts: 5
Joined: 03 May 2020, 13:32

Re: DIY input lag tester.

Post by Piwi » 24 Dec 2020, 07:19

howiec wrote:
23 Dec 2020, 21:48
I was just curious which photodiode so that I could look up its datasheet. Just like with any electrical component there are many parameters that can make a difference depending on the application. But I'm guessing their response times are typically in the ns or low us range anyway which is fine for this.
This has been pretty hard to figure out, honestly. parts numbers are seemingly impossible to have on half of these. I did find https://www.kth .se/social/files/54ef17dbf27654753f437c56/GL5537.pdf which kinda looks like the photoresistors I use, but the quoted response time is 20 to 30 ms, and well, using a LED to test, I get way lower response times than these.
It's also a passive component, just a resistor and a photoresistor, and voltage drops because the photoresistor has less resistance the more light. This guy did some tests on different photoresistors, which isn't really useful for this application, but I found it interesting. https://piandmore.wordpress .com/2016/10/14/photo-resistors-in-depth/
There are also photo transistor light sensor (https://www.adafruit .com/product/2831), I have some of those on order from aliexpress, they have not arrived yet, but I'm curious as to how they behave.
howiec wrote:
23 Dec 2020, 21:48
Yeah, I was also curious about your mounting method to minimize any ambient light effects. Clearly, some sort of shroud would help in this regard.
I do have a bit of a shroud, I took insipration from how Nvidia's LDAT is mounted, with an elastidc band. Before that came out, I used to tape it to my monitor :D
shroud.png
shroud.png (409.03 KiB) Viewed 7476 times

howiec
Posts: 183
Joined: 17 Jun 2014, 15:36

Re: DIY input lag tester.

Post by howiec » 24 Dec 2020, 11:45

Piwi wrote:
24 Dec 2020, 07:19
which kinda looks like the photoresistors I use, but the quoted response time is 20 to 30 ms, and well, using a LED to test, I get way lower response times than these.
Those are definitely photoresistors. Interesting that you seem to get response times different than spec. Regardless, I strongly recommend photodiodes instead for various reasons including much lower response times, and less sensitivity to incident light. Then again, I've never used either so YMMV. I just have an electrical engineering background.

Piwi wrote:
24 Dec 2020, 07:19
There are also photo transistor light sensor (https://www.adafruit .com/product/2831), I have some of those on order from aliexpress, they have not arrived yet, but I'm curious as to how they behave.

...

I do have a bit of a shroud, I took insipration from how Nvidia's LDAT is mounted, with an elastidc band. Before that came out, I used to tape it to my monitor :D
I think they messed up their name because a standard transistor has 3 terminals. It's 99.99% a photodiode (PN junction). Should be an improvement though and although it may not be as sensitive to incident light, a shroud is still recommended unless testing proves otherwise. Hope it goes smoothly!

Post Reply