Potential HD Mouse Extensions API Specification [ETA by 2025]

Talk to software developers and aspiring geeks. Programming tips. Improve motion fluidity. Reduce input lag. Come Present() yourself!
User avatar
Chief Blur Buster
Site Admin
Posts: 8629
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 19 Dec 2020, 16:49

griffin1987 wrote:
18 Dec 2020, 07:48
Would there be a way to reverse everything to use push instead of poll? Like freesync/g-sync kinda does it for monitors.
This could also lead to power savings which could be relevant for mobile users. The mouse would just send movement or clicks when they happen, up to x hz.
Thanks to mandatory timestamps for all mouse coordinates (with a strong recommendation that timestamps be sensor-side) -- the proposed HD Mouse Extensions API doesn't care if it's push or poll or even asynchronous (analog-like).

I've intentionally made HD Mouse API proposal agnostic on poll/push -- and sensor-side timestamping will help benchmark whether or not push/poll is lagged or not (and force vendors to improve that).

HD Mouse API can use non-USB methods. Imagine connecting a Thunderbolt mouse with DMA permission granted, if it can be done in a secure way, with a sufficiently flexible cable! It's a bit overkill for majority, but that would be sort of the Holy Grail to run at direct dedicated PCI-X performance specs, if the cable is not TOO thick.

Another theoretical idea is GPIO based methods, like turning the mouse cable into a dedicated SPI bus to a special PCIe card in the computer. But we likely won't do that, because it's too specialized and would cost several hundred dollars per retail unit just to pay off the engineering/NRE costs. However, any hobbyist designer could theoretically try doing that, if they're familiar enough with creating a custom mouse driver that presents standard APIs (legacy mouse API and future HD Mouse API) to apps.

However, behind-the-scenes, USB is the standard for mice, and high-rate USB is better done as a poll method for lower lag, since push-based USB is laggier. Polls can be continuous (for stationary mice) or only intermittent (movement + button). The problem with intermittent mode is the lag before the first movement/press when a mouse is dormant. Because of behind-the-scene USB sheninigians such as power management and such. Which is why some mice have a force always-poll (continuous 1000 Hz even with stationary mice).

My view is we're stuck with USB for now, and HD Mouse API can help punch through USB performance issues with sensor-side timestamps and poll aggregation (e.g. 20 KHz raw mouse deltas on just 2KHz poll, at 10 deltas per poll). Overkill for 60 Hz days, but important for future retina refresh rates on ultrafast-GtG displays.

And you can still do some USB workarounds like performance tweaks + dedicated USB card + dedicated PCIe lane to CPU (and Razer just mentioned they're discussing with motherboard vendors about guaranteed-dedicated-chipped USB ports with zero contention with any other buses and PCIe lanes, etc).

Regardless, latency purists can skip that and do 1:1 operation with the best mouse connection available, while smoothness purists can reduce poll rate + add about 0.5ms tapedelay (for multiple-deltas-per-poll aggregation overheads) to get massively better mouse smoothness with a lower pollrate. The aim is to let the user choose, while still being plug-n-play.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

       To support Blur Busters:
       • Official List of Best Gaming Monitors
       • List of G-SYNC Monitors
       • List of FreeSync Monitors
       • List of Ultrawide Monitors

Brainlet
Posts: 54
Joined: 30 May 2020, 12:39
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Brainlet » 26 Dec 2020, 13:53

Related to this topic, I also think game devs should adopt sensitivity settings like Diabotical did. When you click on "Mouse Details" you can type in your DPI and the desired cm / 360° (with multiple decimals as well) and it will automatically set the sensitivity multiplier to match it (I don't have the game installed and couldn't find a screenshot of that section).
Starting point for beginners: PC Optimization Hub

User avatar
Chief Blur Buster
Site Admin
Posts: 8629
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 02 Jan 2021, 00:37

Brainlet wrote:
26 Dec 2020, 13:53
Related to this topic, I also think game devs should adopt sensitivity settings like Diabotical did. When you click on "Mouse Details" you can type in your DPI and the desired cm / 360° (with multiple decimals as well) and it will automatically set the sensitivity multiplier to match it (I don't have the game installed and couldn't find a screenshot of that section).
A possibly great crosspost for the HOWTO: Please Future Proof your Game Engine

___________________

On another topic, thought of: Also, poll rate and sensor rate is sometimes a brute-force hammer for the inability to synchronize polltime perfectly to gametime.

Now, the HD Mouse API is to have mandatory timestamps with all mouse deltas (with a strong recommendation that they be sensor-side timestamps). This means the mouse can send timestamps independently of any granularity or interval -- that is now possible with HD Mouse API.

Asynchronous / Analog-like = A mouse that sends deltas that is not time-aligned with a poll Hz. Since timestamps must be no worse than microsecond accurate where possible. Perhaps a time-resolution query is a good idea, so that the time resolution is known -- i.e. like a QueryPerformanceFrequency equivalent.

But the HD Mouse API could provide an additional API call where the game asks for a specific delta nearest a specific timestamp (or current time), and it returns that delta, if it is yet available, and if so, how old/long ago the nearest available delta was. If current time, the internal implementation of the mouse programming could actually do 2-way communication to read that sensor data right there and then, and return the delta -- perfectly synchronized to the game time.

Now, there are potential major problems with that approach:
- Variable USB lag (The Internet atomic time-sync problem, but on a local scale)
- Deltas not yet available (requesting too early = delta not available)
- Delta too old (requesting too late = adds lag)

Another internal implementation vector/curve data transmitted from the mouse sensor to the HD Mouse API, and the HD Mouse API would compute the appropriate delta (along the line of the movement vector/curve) from a timestamp request. So many internal driver workflows can be unlocked by a HD Mouse API internally -- this would be the internal responsibility of the mouse driver. As long as the timestamps-vs-delta are accurate, it could be a legitimate innovative workflow.

However, almost anything in this territory pretty much digitally sampled anyway, albiet at high frequencies, so sometimes the brute is much easier. What we need is to reduce the system load/requirements of such high rate of deltas.

In this sense, it is the lesser of evil to just have a compact USB packet of compressing 20 deltas into one poll (20KHz sensor read in 1KHz poll), all with their timestamps and relay it to the mouse driver containing HD Mouse API. The game engine would just cherrypick the closest timestamp. You would eliminate the need to monitor 20,000 mouse events per second, yet you easily cherrypick the 500 mouse deltas closest to your 500 gametimes for 500 frames per second on a 500Hz+ monitor.

Low workload; doesn't overload CPU or USB chips, you can still configure pollrate to your system's limit but it would not limit the precision of deltas (just latency of a lower poll rate -- but 2000Hz pollrate is only 0.125ms laggier than 4000Hz pollrate from a pure mathematical latency-difference calculation -- half of ((1/2000sec) - (1/4000sec)).

So I think a "delta nearest requested timestamp" API call, is likely a sensible addition. We just need to think this through -- carefully.
- Game calls HD Mouse API asking for mouse delta closest to now (or a specific historical timestamp not too long ago)
- HD Mouse API returns the delta.

This can be done just once per frametime.

(How the mouse driver handles this internally is up to the mouse driver. Most of the time it would be just a small trailing buffer of conventional historical sensor reads, but it unlocks optional internal 2-way workflows with mice connected via consistent ultralowlatency buses: Actually asking for the instantaneous sensor data at the instant)

Actually giving poll responsibility directly to the game software to actually poll the mouse once per gametime -- would eliminate mouse-event load on the software (e.g. 20,000 mouse events per second from a 20,000 Hz mouse), gobbling lots of CPU.

This would not be the only workflow, but it would unlock a hell lot of mouse precision, and also be far more jitter-immune, and permit a more asynchronous mouse workflow -- literally it unlocks an infinite-Hz analog mouse in theory, without the problem of an infinite pollrate.

So 8000 Hz mice would not overload a game engine with 8000 mouse movement events per second. Instead, the game engine would actually call the mouse driver to ask for the current delta.

Many existing game engines also wrappers a mouse and hides the event implementations away from the developer, only returning coordinates once per frame. In that sense, only the core engine needs modification -- the user of the game engine would not need to know the HD Mouse API existed! e.g. Unreal Engine, Unity Engine, etc. The engine vendors would just be responsible for HD Mouse API and everything would continue backwards compatible. Except you get 20,000Hz mouse precision without the CPU load of a 20,000Hz mouse!

The ability to query HD Mouse API only on-demand would do a better service to the Vicious Cycle Effect (higher Hz and higher resolutions amplifying tinier imperfections).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

       To support Blur Busters:
       • Official List of Best Gaming Monitors
       • List of G-SYNC Monitors
       • List of FreeSync Monitors
       • List of Ultrawide Monitors

Post Reply