Interesting project about mouse/ gamepad latency

Everything about latency. Tips, testing methods, mouse lag, display lag, game engine lag, network lag, whole input lag chain, VSYNC OFF vs VSYNC ON, and more! Input Lag Articles on Blur Busters.
spacediver
Posts: 505
Joined: 18 Dec 2013, 23:51

Re: Interesting project about mouse/ gamepad latency

Post by spacediver » 10 Jul 2019, 17:04

ad8e wrote: I can't concoct such a story at the moment. The idea that manual tracking could be better than the other top visual reaction systems by any meaningful amount (even just 10 ms), in improving a reaction test like yours, doesn't agree with my understanding of the human brain.
ad8e wrote:One last thing about manual tracking - while I don't see the potential for Marwan's "manual tracking reflex" to exist, Rejhon's idea that continuous reactions may lower reaction time doesn't have the same story barrier against it. It's pretty easy to find a plausible reason why it might happen, although I would strongly recommend to revise this hypothesis to "reaction tests happening in quick succession will be faster, since the earlier tests prime the later tests".
I'm not wedded to the idea of manual tracking. I came into this seeking to explain an observation. I'm open to any number of possibilities (including my data interpretation being wholly incorrect). Even if kukkii's reaction times were shown to be faster during lg than during simple detection of luminance onsets, it could be a context dependent effect unique to him (e.g. he's able to get in the zone more easily during the former), rather than reflecting a distinct mechanism of action.

But I'd still like to hear more about why you think the idea of manual tracking having a latency advantage over simple detection is implausible. You find the idea of priming, while employing the magnocellular system, plausible. Couldn't a similar story be told about successive responses to changes in visually perceived motion? And what of the idea that some situations may promote a state where recurrent processing is diminished?

Just found this (skimmed study, not sure about quality):
Two-dimensional tracking reveals numerous similarities which exist between eye and manual tracking. Neither system can be adequately described by two independent cases of tracking in one dimension. In addition, both systems appear to use error signals which at some level incorporate both positional and directional error as well as speed mismatch. Furthermore both systems appear to be much less responsive to errors in acceleration (Lisberger et al. 1987). Therefore it is possible that both systems use the same error signals derived from the original retinal error, and may in fact share some of the same trajectory planning apparatus, varying at some point due to the obvious differences in the end effectors.
( https://www.physiology.org/doi/full/10. ... .83.6.3483 )

Is it the idea that manual tracking may use a unique control regime compared to simple responses that you find implausible? Or the idea that even if such a system existed, that it would confer a significant latency advantage?

Onto the data:

Image

I agree that a good model of lg performance would be wonderful to have (where reaction time could be one of the fitted parameters), and you've certainly made me question the validity of my interpretation, however here's a stab at defending the cross correlation.

I've highlighted a single point in the enemy accel waveform (cyan dot) and a single point in the xhair accel waveform (green dot).

The cyan dot indicates (roughly) the moment at which the dodger (me) has initiated the direction change (I'm assuming that quake models directional inputs as forces upon the body). The green dot represents roughly the moment at which ku applied a change in force direction to his mouse.

First note that the number of major "force" events is matched between enemy movement and mouse movement. This alone is suggestive that the forces applied to the mouse are informationally related to the forces "applied" to the dodging enemy. It's only suggestive, since the same result could occur with random wobbles, so long as those random wobbles had the same approximate frequency as the dodging frequency. But I consistently saw that these force events were separated by a similar amount.

If random wobbling was responsible for low lag times in the cross correlation (and you nicely explained how this could occur), then you'd expect to see a uniform distribution of lag times, whereas in the four frags I analyzed, they were fairly close. Moreoever, if random wobbling was responsible for this, you'd expect a low lg accuracy, when in fact he was performing quite well. Put another way, we already know there was a degree of intelligence and responsiveness at play here, and can therefore have more confidence that the features of the signal that account for the most variance are, in fact, meaningful (i.e. they are in response to the enemy's movement).

spacediver
Posts: 505
Joined: 18 Dec 2013, 23:51

Re: Interesting project about mouse/ gamepad latency

Post by spacediver » 10 Jul 2019, 17:04

note: I won't be able to respond until later tonight or tomorrow.

ad8e
Posts: 68
Joined: 18 Sep 2018, 00:29

Re: Interesting project about mouse/ gamepad latency

Post by ad8e » 10 Jul 2019, 22:08

spacediver wrote:I'm not wedded to the idea of manual tracking. I came into this seeking to explain an observation. I'm open to any number of possibilities (including my data interpretation being wholly incorrect). Even if kukkii's reaction times were shown to be faster during lg than during simple detection of luminance onsets, it could be a context dependent effect unique to him (e.g. he's able to get in the zone more easily during the former), rather than reflecting a distinct mechanism of action.
If someone were really able to train himself to that extent, that would be pretty interesting. I know how such a training might be plausible, but seeing it is another thing.
spacediver wrote:But I'd still like to hear more about why you think the idea of manual tracking having a latency advantage over simple detection is implausible. You find the idea of priming, while employing the magnocellular system, plausible. Couldn't a similar story be told about successive responses to changes in visually perceived motion? And what of the idea that some situations may promote a state where recurrent processing is diminished?
I chose luminance as my go-to default, but I don't have any reaction-related knowledge here. My experience is only that the human mind's luminance processing seems to be higher performance than its color processing in tasks such as reading. A combined luminance/hue change would be better, as it avoids this confounding variable. Blue to bright green combines both a huge luminance change and a nice color change, since blue is naturally "dark". (This is in reference to his mention of the magnocellular system.)

Priming could exist in reactions during motion detection, but primed motion detection should lose to the equivalent primed luminance-change detection; the priming should be a general mechanism if there. Motion detection is hard, and the human mind is very good at it, but it can only be good to a limited extent since the underlying problem is difficult to deal with. But detection of giant changes is very easy, the human mind is good at it automatically. Since I don't believe priming is specific to motion, I then naturally want to use a switch-background test, since those are nicer than motion tests.

Reduced recurrent processing: yes, that's one of the possible reasons why faster reactions might exist. For short-term differences (i.e. tests under various conditions), rather than long-term changes (i.e. tests 5 years apart), it's a strong explanation that is considered when investigating every type of faster processing. In other words, reduced recurrent processing is an excellent story, and one that people look to frequently.

Motion vs luminance: imagine a black square moving on a white background and then switching movement directions; this is motion. Then imagine changing the test so that instead of switching directions, the black square instantly fills half the screen on its trailing edge. Directly comparing the two situations, every change in the motion test also exists in the fill test, so under a more general theory of cognition than the human mind, the fill test should have only advantages over the motion test. Applying this general theory of cognition directly to the human mind is wishful thinking, since it's quite likely that the motion detection system of the human mind will fail to produce a useful reaction and shut down when it detects the screen being filled. Then, some other part of the brain will be needed to provide the reaction. But the moral is still there.
spacediver wrote:Just found this (skimmed study, not sure about quality):
Two-dimensional tracking reveals numerous similarities which exist between eye and manual tracking. Neither system can be adequately described by two independent cases of tracking in one dimension. In addition, both systems appear to use error signals which at some level incorporate both positional and directional error as well as speed mismatch. Furthermore both systems appear to be much less responsive to errors in acceleration (Lisberger et al. 1987). Therefore it is possible that both systems use the same error signals derived from the original retinal error, and may in fact share some of the same trajectory planning apparatus, varying at some point due to the obvious differences in the end effectors.
( https://www.physiology.org/doi/full/10. ... .83.6.3483 )
The scholarship and experiment design of that paper are superb. Their latency measurements are suspect; I think they didn't realize that a 100 Hz touchscreen and 60 Hz monitor might have much more latency than caused by 1/Hz, which is why they didn't measure it. But they mentioned the exact models they used, so their experiment is replicable, and this omission doesn't affect the thrust of their results. Here's my understanding of the article's relevant takeaways:
1. Motion detection is hard, position detection is much easier. "after the target changed direction, the finger maintained the original target direction for a reaction time period, changed direction, initially headed in a nearly straight line to intercept the target, and then finally curved to merge with the new target direction" That's consistent with position detection being faster than motion detection. By the time the person realizes the position has changed, he still is unable to estimate velocity. This doesn't apply to kukkii's situation exactly, since in Quake, velocity changes in consistent ways and can be estimated through training. In the paper, the people have no training and the velocity change is random.
In more casual terms, this means that Quake's timenudge is an objective advantage and the difference can't be closed by training. This is exactly what is predicted by cognition, and it's nice to have a specific paper to point to for this specific effect, instead of me saying "trust me, I'm an expert", which nobody should take seriously.
2. Acceleration detection is really hard. "partial information about acceleration is present only in the population response of these neurons". In laymen terms: the brain sucks at it, even after considering the task being hard. This is opposed to motion tracking, which is also a hard task, but one the brain tries to be really good at.
3. Training will significantly improve performance on the task in the study, since the constant-time reacquisition behavior they modeled is suboptimal. This provides a clear demonstration of a specific performance gap that training can reduce, and similar considerations should also apply to Quake.
4. Reaction times to changing speed are about the same as reaction times to changing direction.
5. Their model of constant time to reacquisition is quite unusual, and I wonder why it happens. I didn't expect that.
6. When the finger tries to track the box, the finger's speed is slower. Not just a offset, but gradually falling behind. I'm not sure why.
7. Looks like the authors are as surprised as me about constant time reacquisition: "We did not anticipate this result. It is entirely possible that the time to intercept could have been minimized"
8. Their results show similarities between eye tracking systems and positional tracking systems, and imply that there may be shared machinery between them. This is despite the split of eye tracking into smooth and saccadic movements.
spacediver wrote:Is it the idea that manual tracking may use a unique control regime compared to simple responses that you find implausible?
That's not just plausible, it's guaranteed to be true, and I recall some details of the mechanism. I think even the specific responsible parts of the brain have been isolated, but I'm not clear about these things.
spacediver wrote:Or the idea that even if such a system existed, that it would confer a significant latency advantage?
Yes, that's it. I don't think manual tracking can give a latency advantage. Specifically: motion detection is hard, and won't go faster than a simple system. Also, trying to track an object with your hands is not a mechanism that will be the direct cause of improved reaction times, in the sense that if there's an improvement in reaction times, it is caused by some other part of the action, such as eye tracking, or hand motion irrespective of tracking, or tensed muscles, something like that.
spacediver wrote:Moreoever, if random wobbling was responsible for this, you'd expect a low lg accuracy, when in fact he was performing quite well.
The LG accuracy won't decrease, the optimal aiming model has something that is very similar to the wobbling. (I think it's: while tracking the target, pretend that the target switches direction at a specific point in time in the past, and then track the new phantom trailing edge in the opposite direction after it intersects the existing aim path.)
spacediver wrote: Put another way, we already know there was a degree of intelligence and responsiveness at play here, and can therefore have more confidence that the features of the signal that account for the most variance are, in fact, meaningful (i.e. they are in response to the enemy's movement).
Eh, I think it will be hard to convince me here. I don't see a way to give a good demonstration of the integrity of the analysis, short of doing some statistical modeling and providing some hard numbers, and I'm not able to do that myself.

For example, one of the properties of the wavelet detection code I want is that it has to operate in real-time. Note that Pain and Gibbs don't use a real-time detection algorithm, they detect it retroactively. But then they have to do some statistics to prove that their detection isn't cheating and capturing some random noise as the beginning of the wavelet. Even then, it's a little sketchy. By doing real-time wavelet detection, the need for these statistics is removed, and the integrity is built-in, since any mistakes will be captured as false starts.

(Note on the forum software: when writing responses that take a long time, copy them to the clipboard before pressing preview or submit, because the forum often trashes them. I've avoided this problem by an abundance of caution, but if I wasn't so cautious, I think 3 of my responses would have been killed.)

spacediver
Posts: 505
Joined: 18 Dec 2013, 23:51

Re: Interesting project about mouse/ gamepad latency

Post by spacediver » 11 Jul 2019, 00:35

Gave a read through of your latest post - great stuff, this is a good discussion, and thanks for the summary of that paper!

I'll try to write more before the wknd, but if I'm unable to, don't take that as a lack of interest!

Brief comment on the reentrant stuff and how it might relate to manual tracking:

recurrent processing seems to be involved in converging upon a perceptual hypothesis, with interactive contributions between downstream and upstream brain regions. In a situation where we don't expect the target to change, it may be more efficient for the visual system to trade a degree of perceptual confidence for a latency advantage (i.e. brain says - hey, we already have a pretty good idea of what this object is, let's stop this constant "recapitulation" of the target, and just go with the flow).

Re position vs motion, one counterpoint is that manual tracking performance will still benefit in the absence of quality velocity information. All the brain needs to figure out is when to initiate the direction change (and perhaps in what general direction), and that alone will facilitate performance. In other words, if the target changes direction, and you don't, you will be worse off compared to if you do change direction, even if your velocity is a bit off (although I haven't groked the optimal aiming model yet, so this might be wrong).

and yes, it's tragic losing posts, I've been using notepad as a temp backup for a while now. Good call.

I have some experience developing real time algorithms, I might be able to implement the wavelet detection code if I study the idea more carefully. Would this be simply for peak detection?

ad8e
Posts: 68
Joined: 18 Sep 2018, 00:29

Re: Interesting project about mouse/ gamepad latency

Post by ad8e » 11 Jul 2019, 03:34

If we assume Overwatch physics and the Quake AABB collision box, then the aimer doesn't need to know the dodger's velocity, only his history of positions (which does determine velocity, but let's ignore that). The aimer pretends that his opponent only uses A and D, so it's important to detect the moment of velocity changes and that's all that's needed. If the opponent doesn't use A and D, the aimer nevertheless continuously assumes that the dodger will change his mind and start using A and D immediately, even as the aimer is continuously proven wrong. Acceleration-based A and D, as in Quake, is different and the aimer needs to read velocity; being able to tell that acceleration changed is the same as being able to estimate velocity accurately. Timenudge and leaning characters change the analysis, but they push the overall picture of what's happening away from the concept of motion tracking.

Here's the model for wavelet detection to be considered, for a code that takes in a sequence of (mouse X, mouse Y, time):

1. do0om moves his mouse downward. It might be at a slow pace or at a medium pace, but it is consistently nonzero. You should expect this to be pretty jerky in speed within a single motion. The direction won't be perfectly vertical either, but direction will be relatively consistent within a single motion.
2. At some point, his mouse will jerk leftward. At the earliest possible time, decide when it's a true jerk leftward instead of random noise from the downward motion. This needs to be conservative enough not to introduce false positives, but it needs to be very aggressive, being able to say "it's now!" nearly at the beginning of the acceleration. A sophisticated approach is needed, and you'll need to tune the parameters on your own hand. A good idea here is to record a bunch of mouse movements and jerks, then test various algorithms on the stored data.
3. the mouse is 1000 Hz but some timepoints may be missed, or some timepoints may be off by a little. You'll be able to see by how much when measuring your own mouse.

His sensor is probably flawless, so it'll be similar to your mouse which is also probably flawless. His hand will be somewhat different. We'll ask him to use a reasonably high CPI, one that his mouse is still able to handle without smoothing. Something like 5000-10000 sounds ok, but it depends on his mouse's capabilities.

Are you familiar with C++? What about glfw? I can send you mouse recording source code or just a binary.

spacediver
Posts: 505
Joined: 18 Dec 2013, 23:51

Re: Interesting project about mouse/ gamepad latency

Post by spacediver » 11 Jul 2019, 09:42

I've developed algorithms in C++, not familiar with glfw. Send binary (I'll pm you email address).

I use an old mouse (deathadder 3.5 g sensor, optical DPI of 450, inside a wingman gaming mouse shell). I also have a cheap optical (cooler master devastator 2) that has a mousewheel for when I need it, might be better than the deathadder's sensor, will have to check.

User avatar
Chief Blur Buster
Site Admin
Posts: 11648
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Interesting project about mouse/ gamepad latency

Post by Chief Blur Buster » 11 Jul 2019, 10:23

ad8e wrote:(Note on the forum software: when writing responses that take a long time, copy them to the clipboard before pressing preview or submit, because the forum often trashes them. I've avoided this problem by an abundance of caution, but if I wasn't so cautious, I think 3 of my responses would have been killed.)
spacediver wrote:and yes, it's tragic losing posts, I've been using notepad as a temp backup for a while now. Good call.
Thankfully, a planned forum software upgrade is coming this year.

The upgrade -- which will provide better editor autosave -- will will also provide simplification options for users (merging accounts on BlurBusters.com comments & forum login) and a quick social-media-connect registration process, while keeping traditional hard-captcha-powered registration for users who don't want to connect via social media.

Other than that catch -- I'm just lurking (for now) in the constructive discussion/brainstorms. Given my well-known long-winded posts, I don't want to throw too many distracting Pandora Boxes in the middle of things...
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

do0om
Posts: 34
Joined: 15 Jan 2018, 13:14

Re: Interesting project about mouse/ gamepad latency

Post by do0om » 11 Jul 2019, 12:10

I have a g502 mouse, 12 000 dpi is the max. But I think I read somewhere that there is smoothing on high dpi.
By the way, Rocket jump ninja posted this : https://twitter.com/RocketJumpNinja/sta ... 6433448966 (Maybe he is watching the post :D ).
147 ms on 25 tries, this is pretty damn good.

ad8e
Posts: 68
Joined: 18 Sep 2018, 00:29

Re: Interesting project about mouse/ gamepad latency

Post by ad8e » 11 Jul 2019, 12:34

do0om wrote:I have a g502 mouse, 12 000 dpi is the max. But I think I read somewhere that there is smoothing on high dpi.
By the way, Rocket jump ninja posted this : https://twitter.com/RocketJumpNinja/sta ... 6433448966 (Maybe he is watching the post :D ).
147 ms on 25 tries, this is pretty damn good.
Pretty cool. Given what I know of your setup, your humanbenchmark test results can be reduced by 9.5 ms through software changes, with 4 ms of that coming from a different mouse button, and the other 5.5 ms coming from screwing your browser over by uncapping its vsync. Still, RJN's result is better, because it doesn't leverage the specific advantages I know of, and because its 25 trial count is much bigger.

With a new setup though, better quality data is coming, and afterwards we'll no longer need to reference the humanbenchmark results. It doesn't even have a false start condition, the 100 ms threshold is applied after averaging, so players can guess 0 ms at the beginning and start the test after that.

I think the G502 has no smoothing changes over DPI, so we'll try setting your DPI at 6400.

flood
Posts: 929
Joined: 21 Dec 2013, 01:25

Re: Interesting project about mouse/ gamepad latency

Post by flood » 11 Jul 2019, 13:11

been a while since last posting here... spacediver alerted me to this discussion

1. my old 148ms is not fake and not cheating. it did not have an outlier that was less than 140ms; i wouldn't report the average if i just got lucky...

i tend to cluster between 150ms and 170ms. sometimes with outliers above if the timing catches me by surprise. very rarely I'll accidentally misclick and get a time less than 140ms.

2. idk about the current reaction time test, but a long time ago i checked against my own reaction time test program running at 1000fps and didn't get much different results from the humanbenchmark site (which was running flash)

3. i've probably made a similar comment before, but evaluating tracking latency is tricky due to prediction... also the goal there is not to react as fast as possible but to keep your crosshair on the enemy

Post Reply