Potential HD Mouse Extensions API Specification [ETA by 2025]

Talk to software developers and aspiring geeks. Programming tips. Improve motion fluidity. Reduce input lag. Come Present() yourself!
User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 08 Nov 2020, 00:49

Hello,

This is just a scratchpad for now;

Lots of Blur Busters stuff incubate from Laboratory area (including Area 51) which are higher % industry-lurkers (large lurk readcounts from businesses) rather than from younger fans in the General/Latency area. Over 99% of non-bot forum visitors are lurkers, and clear workday bandwidth traffic comes from Asian lurkers (i.e. the manufacturers). Eventually this will promote to an official Blur Busters page, but for now, this is just scratchpad stage. Currently inspired by this Twitter Thread between myself and Razer.


High Definition Mouse Extensions API Proposal

I've been doing this for some time, but I've finally got some things rolling that might happen by year 2025. This is my idea, not any mouse vendor's idea. Yes, I'll run with the first vendor to bite, and no, I'm not asking for money or rewards for this. And no, Razer didn't request this. But because of improving industry connections, I'm now pushing for this. It's part of my passion.

Things are now far enough along that I'm throwing a trial balloon into a piranha tank (online, offline, behind scenes, forum). This is not a real spec, this is just a discussion of a possible spec to eventually gel. The goal is something by year 2025 because we are reaching severe mouse limitations in the refresh rate race to retina refresh rates.


Introduction

I have years of experience collaborating with standards authors, such as XEP-0301 In-Band Real Time Text which is used by some projects, and was adopted into some Next Generation 911 systems (so deaf can type to 911 operators, and 911 can their message typed one character at a time, lest they get interrupted/die mid-way typing) -- my XEP-0301 standard mentioned at fcc.gov. Although they ultimately adopted RFC4103 instead, it does underscore my standards experience, and the message-by-message version of RFC4103 actually borrowed some ideas later already. But I write this just to mention how beautifully formatted and RFC2119-compliant a spec document I can write.


Higher Display Refresh Rates Expose Major Limitations Of Mouse

Higher resolutions and higher refresh rates, with clearer motion (from refresh rate or strobing) amplify human-visible mouse microstutter. New 360 Hz monitors show the limitations of 1000Hz poll rate much more visibly than at 240Hz and 120Hz. Part of the reason why people in LinusTechTips YouTube video could not tell apart 240Hz versus 360Hz is because the mouse poll was only set to 1000Hz. This diminished the human-visible difference between 240Hz and 360Hz, because the difference was obscured by mouse jitter. Despite other Amazing Human Visible Feats Of The Millisecond.

Current mouse architecture hurts the market for all high-Hz displays, since the latest mouse sensors are now capable of far better precision than currently exposed to the computer/games. This is the main raison d'etre of a High Definition Mouse Extensions Specification


Problem
  • 360 Hz monitors have recently showed limitations of 1000 Hz mice
  • Many esports gamers want to stay at 800dpi
  • High poll Hz can create undue system load
  • Turbo mouse cursors occur with high DPI
  • The need to be compatible with legacy games
  • The need for more precision in newer games
  • Different gamer preferences in DPI and sensitivity
  • Low latency use cases versus smoothed use cases
  • The need to stay microsecond jitter accurate in increasingly high-level game programming (C# C++ HTML5 etc)
  • The need to be compatible with ultra low latency for pure C/ASM/RTOS implementations
  • Current mouse architecture, derived from The Demo (Doug Engelbart), is increasingly problematic
  • Becomes problematic in the light of Vicious Cycle Effect
  • Other conflicts between different stakeholders (users, vendors, OS, etc)
Example 1: Gamer reduces mouse DPI to 800dpi for best CS:GO performance.
Example 2: Another gamer sets 3200dpi (low sensitivity such as 0.06) for best Valorant performance but gets turbo mouse cursor in menus or exiting to Windows
Example 3: DPI switching manually configured by the user on a per-game basis in mouse vendor software
Example 4: Some games gets affected by the Sensitivity slider in Windows
Example 5: 8000 Hz pollrate solves some problems but creates new problems
Example 6: Gamer permamently leaves mouse at 400dpi, limiting mouseturn resolution


Ideal Goals

A new high definition mouse API that is more future proof, yet isn't too onerous for all mouse vendors and software developers.
  • Backwards compatible
  • Allows 400dpi and 800dpi to continue to be used
  • User friendly
  • No cursors too fast for users
  • Way to avoid pollrate overloading CPU
  • Newer games gets fuller access to mouse sensor rate
  • Less frequent DPI switching

The Details
  • Maintains USB HID and maintains default behaviour
  • Yet newer games get full automatic access to full precision via new parallel API
  • Full access to full sensor rate without poll rate granularity interference (e.g. direct 20 KHz sensor access)
  • Users can stay at 800dpi yet newer games still have access to full sensor resolution
  • More microsecond accurate; easier for developers to dejitter mouse
  • Compatibility with connecting a mouse via a different cable more time-precise than USB (e.g. SPI, GPIO) or
  • Compatibility with preserving full sensor rate even at low poll rates via poll batching with mouse-side microsecond timestamping of sensor reads
  • Reduce the CPU overhead of 8000 Hz or 24000 Hz poll rates
  • Maximum backwards compatibility with USB HID
  • Easy for software developers who choose to support extended API
  • Extended API can be turned OFF, we don't want to force this upon all users
  • No medium-enforced upper limit to poll rate/sensor read rate, e.g. 20KHz sensor reads can be batched at 20 timestamped coordinates over 1000Hz USB
To get multiple vendors to agree to the same spec... It will still be important to allow mouse vendors to compete against each other by specializing in various tweaks to HD Mouse Extensions API, such as being the first to offer customizable smoothing windows that trade latency versus smoothness, like allowing users to configure "I want to batch 2000Hz/4000Hz timestamped polls into 1000Hz poll, with no more than a 0.25us dejittering time window" (via a wizard or settings screen) to other things compatible with HD Mouse Extensions API, or other things that are not more CPU-heavy than 1000Hz The HD Mouse Extensions API should be highly extensible in a future proof way, with the ability to expose capability in a more transparent less-blackboxed way. So it's possible there will be limitations such as not being able to cram 20 KHz sensor reads into 1 KHz poll rate on one vendor, but is possible on a different vendor. But the new HD Mouse Extensions API Protocol should allow such flexibility.

There are multiple potential routes of a high definition mouse extensions API, but the common goal is to get multiple mouse vendors to agree to a common extended mouse API for futureproofing the refresh rate race to retina refresh rates. An open, royalty free, patent free specification. But the route there have different pros/cons that make it easier/harder to meet the ideal. So specifications are inherently compromises. A Version 2 spec (Some ideals) that has a path to Version 3 spec (all ideals) is ideally preferred, so we don't want to architecture ourselves into a corner.

Thus, it is possible not all ideal capabilities may be included in the initial spec, however, the specification should remain extensible.


High Display Refresh Rates Need Both High DPI and High Poll Rates

High DPI fixes slowturn jitter
If you do 400dpi + 4.0 sensitivty, your slowturns are steppy which sabotages slow-movement fluidity/framerate. But you do 1600dpi + 1.0 sensitivity, slowturns are much smoother. You probably already know that. But many games are crappy at high-DPI settings, so people stay at 400dpi or 800dpi. Now even if you use 800dpi+0.5 sensitivity, remember even a half-pixel mouseturn is still human visible in some games (it looks like an antialiased mouseturn) because 3D can be rendered subpixelly. So you want subpixel precision headroom for motion nirvana, assuming the game co-operates properly with high DPI without smoothing/interpolation sheninigians.

High Poll fixes fastturn jitter
Try doing 125Hz or 500Hz mousepoll on a 240Hz monitor. It feels strangely coarse/jittery during fastturns especially when you've got blur reduction turned on (DyAc ON for example). Raise mouse to 1000Hz and the fastturn jitter is gone. Now the new monkey wrench, <Cue One More Thing Music>, raise your display refresh rate to 360Hz or higher, and make sure your framerate is in that territory. The 1000Hz fastturn jitter faintly comes back in some situations in some games (via watching the stroboscopic effect in-game with your eye on high-contrast edges in-game). Ouch. So the display Hz is now too close to mouse poll Hz and no longer human-invisible. Time to punt up the mouse pollrate a smidge to max out human benefit at least in the visibility department. 2000Hz at least, to avoid the overload factor.

Subpixel Mouse Turns Are Human Visible
Even a 0.5 pixel mouse turn can be human visible, it just looks like an antialiased mouse turn, since 3D graphics can be rendered at subpixel levels. Valorant does this well, while CS:GO is not as good at this. That's why Valorant works well at higher DPI than CS:GO does (in other words in Valorant, 3200dpi 0.06 sensitivity feels identical to 400dpi 0.48 sensitivity, for fast flick turns, on a mouse without smoothing for 3200dpi. Except now slowturns now benefit).

Need To Stay 400dpi and 800dpi Compatible For Legacy
Many legacy games such as CS:GO do not behave very well at 3200dpi. Even when the sensor has smoothing/interpolation/etc disabled, 3200dpi at 1/4th sensitivity you use at 800dpi -- don't necessarily behave the same because the old engine didn't scale well for high-DPI operation.

Easier Preservation of Math Precision for Mouse Coordinate Mathematics
I even now recommend software developers to cast (int) mouse coordinates to (double) instead of (float), as a higher resolution intermediate processing step (explained in HOWTO: Please Futureproof Your Game Engine). Much like the 10bit color processing step from an 8bit GPU to an 8bit panel, is still worthwhile (less color rounding errors). But that's a programming step that developers have to actively remember to do. High Definition Mouse Extensions API will help automate the preservation of precision for software developers.

Long Term Benefits to Windows/Linux/Mac Scrolling

Full DPI information for scrollbar scrolling is beneficial. For example, you load a tall webpage/document/code, and try to drag a scrollbar by 1 pixel? The webpage scrolls by multiple pixels at a time. Ouch. But if you could access 12800dpi information while staying at 800dpi mouse pointer speed, the scrollbar would gain 16 subpixel steps per 1-pixel scrollbar movement. Drag-scrolling becomes much smoother and more pleasant, especially on a high-Hz display -- reading smooth scrolling text on a CRT-motion-clarity display makes finding text during scrolling faster.

Scrollbar widgets supporting HD Mouse API would massively smooth scrolling for software that implemented the HD Mouse API compatible scrollbars -- tall texts would scroll much more smoothly on high-Hz displays.

Biggest benefit will be games; that said HD Mouse API benefits UX more than just games.


Possible Proposals For Mouse Transmission

These are just musings of possible routes to achieve the ideals, with respective pros/cons. This is the behind-the-scenes communication channel, not what is visible to games.

1. Mouse Side: Pure HID Protocol Only

1. Mouse sends full resolution coordinates in existing HID protocol (e.g. 12800dpi without smoothing/interpolation)
2. Mouse driver would handle the divisors (e.g. divide 12800dpi numbers by 16 to get 800dpi numbers)
3. All existing mouse APIs stay legacy compatible, with same old fashioned DPI switching behaviours
4. The magic is that a new HD Mouse API becomes available to newer games, accessing the full original DPI

Pros: Works out-of-box with Windows 10 mouse driver. Less involvement from OS/Microsoft since it fits better in bog standard USB HID.
Cons: Less extensible to meet various specific ideals (some can be done via HID sidechannels, like those used to signal changes to DPI/poll/etc). More research on sidechannel abilities is needed, to see what can be done within the confines of the standard Windows 10 mouse driver (e.g. hidden extended information attached to each mouse poll, which is already being done, this could be piggybacked upon -- but more research is needed)

2. Mouse Side: Custom Protocol

1. Mouse concurrently sends standard USB standard HID packets and high-definition packets
2. Without installing mouse drivers, Microsoft only sees the USB standard HID packets
3. With installing mouse drivers, the extended packets containing extended information can be processed.
4. The extended packets would only be handled by newer games that accesses the HD Mouse API

It is also theoretically possible two separate USB devices appear when plugging in the mouse, but the hub approach needs to be avoided where possible, due to latencies.

Pros: Much more extensible, such as poll-batching and poll timestamping. For example sending an array of multiple separate mouse-firmwareside-microsecond-timestamped sensor reads in one single USB packet, allowing 20KHz sensor via 1000Hz/2000Hz/4000Hz/8000Hz poll even at user configurable thresholds that accomodates user preferences of lag/precision/etc. Extended packet format can be formatted in a way in a USB HID sidechannel compatible way *and* non-USB methods (e.g. SPI, GPIO, Thunderbolt alt modes, etc)
Cons: Massive increases in complexity of hardware engineering, due to protocol, but much more extensible.


Possible Proposals For Software Processing

This is what becomes visible to software that wants to access the new HD Mouse API.

- The HD Mouse API would have metrics to inform the game (sensor rate, poll rate, DPI, etc)
- The HD Mouse API would provide full precision information either as integers or as floating point
- The HD Mouse API provides optional poll batching to reduce system load
- The HD Mouse API can fire additional events for special events (DPI changed, mouse lifted from surface, etc)
- The HD Mouse API can have API-accessible configurability such as poll batching, smoothing ON/OFF, smoothing-vs-lag windows,
- The HD Mouse API will include floating point timestamps (double) or monotonically increasing timestamps (no less accurate than 1 microsecond accurate) for each coordinate pair and button events, with timestamps recorded at sensor level.
- The HD Mouse API will inform application where the timestamps came from (sensor side versus driver side), to allow applications to trust the integrity of the timestamps.
- The game would handle the data as it wishes, ideally with:
......Separate sensitivity adjustment for mouse pointer for menus/inventory/etc
......Separate sensitivity adjustment for mouse turns
......Separate sensitivity adjustment for scopes
Sensitivity adjustment MUST (RFC2119) have at least 4 decimal digits, as explained at HOWTO: Please Future Proof your Game Engine

Optional poll batching reduces system load. To gain sensor rate higher than USB rate (e.g. 20KHz "poll" via 2KHz USB poll via a single USB packet transmitting an array of 10 sensor reads and accompanying timestamps recoreded at sensor level), to allow newer games to access sensor read rates without the overload associated of high poll rates. The poll rate controls the latency you want but you can't just always reliably have jitter-free higher poll rate on some systems. Poll batching should be optional (i.e. a checkbox)

Also, all this can be combined independently with above possible proposals for mouse transmission (e.g. 1+A, 1+B, 2+A, or 2+B). This is only for the HD Mouse API only accessible to newer games. This does not affect legacy games which would continue to get the old-fashioned ordinary behaviors (behaving identically as before).

A. Integer Mouse Coordinates

Full precision numbers would be provided, i.e. if the mouse is 12800dpi, the numbers provided will be always 12800dpi (example) in the HD Mouse API even while the legacy mouse APIs concurrently gets 400dpi or 800dpi numbers (at user dpi preference, as usual). The game will just divide as necessary via high-precision sensitivity adjustments. In other words, permanent new-game access to full mouse precision becomes available if user enables the HD Mouse API.

Pros:
- Simpler for engineering mouse
Cons:
- Requires more work/effort by developers
- Turbo mouse cursor may happen more often by accident, user unfriendly

B. Floating Point Mouse Coordinates

*** Blur Busters Preferred Route ***
The mouse drivers would divide the integer data by the appropriate divisors in the mouse driver before exposing them in the HD Mouse API, without rounding off. Users would configure their preferred DPI as usual, like 400dpi, 800dpi, like always. The bonus is on a 12800dpi mouse (non-interpolated/non-smoothed) "400dpi" through "12800dpi" would preserve same precision thanks to the decimal digits. No legacy wonky mousefeel. It would be integer coordinates for the legacy API, and floating point coordinates for the HD Mouse Extensions API -- identical when rounded to the nearest integer.

Pros:
- More user friendly. Avoids the turbo cursor effect. You're getting "800dpi" numbers with 12800dpi (or any DPI) precision
- Infinitely upgradeable. You can get "400dpi" or "800dpi" from any DPI (such as 12800dpi) with same precision of max DPI
- Fewer separate sensitivity adjustments needed for users; same setting for mouse turns and mouse pointer is easier.
- More flexible to future nontraditional sensor workflows (true asynchronous sampling, vector sensors, etc)
- API will return (double) coordinate deltas to encourage good developer habit to keep higher precision intermediate math processing to reduce rounding errors.
Cons:
- Minor amount of game developer education to explain benefit of floating point mouse coordinates


Alternative ideas that are feasible to manufacturers

There are other possible avenues, but the important thing is that the specification needs to remain relatively simple. Complicating the specification more means more involvement by more vendors (e.g. OS/Microsoft, USB chipset makers, etc) so it has to be designed in a way that minimizes changes to the mouse chain (while adding mouse firmwareside microsecond timestamping of polls + adding optional batching support) and stay easy for users. A user needs to be able to plug in a mouse, and it just automagically works.

*** Please note, this is an early pre-specification napkin proposal of a discussion ***
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

Kiwii
Posts: 1
Joined: 08 Nov 2020, 03:25

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Kiwii » 08 Nov 2020, 03:33

will keyboards with similar 8000k+ polling rate have similar benefits to mice with similar polling rates in games like DOTA2 and LoL where button press latency is important?

Do you expect that we will get keyboards with such high polling rate once we start getting mice with similar specs? Based on your experience how far are we from a product like that?

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 08 Nov 2020, 04:30

(Ooops, fixed some of the bad links, particularly to my XEP-0301 Specification, sorry about the URL typos. I was typing this post up very fast at my full keyboard typing speed.)
Kiwii wrote:
08 Nov 2020, 03:33
will keyboards with similar 8000k+ polling rate have similar benefits to mice with similar polling rates in games like DOTA2 and LoL where button press latency is important?

Do you expect that we will get keyboards with such high polling rate once we start getting mice with similar specs? Based on your experience how far are we from a product like that?
While this thread is not the place for high-pollrate keyboards (and may be split to separate thread) there are definite benefits to lowering keyboard latency but it's also jitter between physical and electronic that is the bigger latency issue actually. Keypress actuation is a bigger weak link which optical keyswitches help address. A 8000Hz keyboard produces much more accurate keypresstime:photontime sync, with only 0.125us jitter, only 1 pixel of misfire per 8000 pixels/sec.

Many low-Hz USB keyboards can be bad at accurate event timing, e.g. firing a secondary weapon too early or too late. A 125Hz keyboard means that keyboard events can jitter up to an ~8ms amplitude, creating a potential-miss error margin for a lot of precision things like using keyboard to precisely time a fire a secondary weapon at a moving target. Most players fire via mouse, but there are some games where you operate secondary weapons with the keyboard, and those benefit particularly well from high-Hz keyboard ability to have much tighter sync of physicalpress:photontime.

One huge problem is two 8000Hz devices on the same computer will create some system loading problems unless you carefully optimize the system. 1000Hz already itself makes an improvement in those games that rely somewhat on the keyboard precision, so 8000Hz is extremely incremental (far less important for keyboards than for mice). Precision improvements for keyboards is a totally different universe than for mice since physical actuation is a bigger weak link, and the transition from mechanical switches to optical switches is a much bigger improvement than 1000Hz vs 8000Hz for keyboards.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

NDUS
Posts: 71
Joined: 12 Aug 2019, 16:05

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by NDUS » 08 Nov 2020, 18:48

Chief Blur Buster wrote:
08 Nov 2020, 04:30
the transition from mechanical switches to optical switches is a much bigger improvement than 1000Hz vs 8000Hz for keyboards.
This is a cool writeup about keyboards, but I just wanted to add that there is now a third option, which is hall-effect switches

eg. https://wooting.io/wooting_two_he

Also, Corsair's K100 keyboard claims to support 4khz polling. Don't know if it's real 4khz or four packets at a 1khz polling rate fakery (like Cougar Minos & Asus ROG Gladius "2000hz" mice)
I agree with you that polling rates above 1khz are not important for keyboards.
Some people with "1khz" keyboard don't even realize that they are actually 60hz keyboards, because they have a 60hz scan rate.

Re: a mouse movement API suitable for 3D applications, this sounds amazing. The sister API to WM_MOUSEMOVE that should've already been made years ago, imo.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 08 Nov 2020, 19:02

NDUS wrote:
08 Nov 2020, 18:48
Also, Corsair's K100 keyboard claims to support 4khz polling. Don't know if it's real 4khz or four packets at a 1khz polling rate fakery
It'd be interesting. This means the difference of 0.25ms vs 1ms jitter between eventtime:photontime. Ideally I'd like too timestamp keypress actuations at the keyboard microcontroller level, so that it's jitterproof. Bypass inaccurate OS timestamping of poll reads.

But initial efforts need to be focussed on bringing a HD Mouse API first to reality. If none of the big ones does it, this stuff is achievable by hobbyist indies on a Kickstarter budget too since some USB2-ported Arduino clones already can do 8000Hz already and can concurrently emulate a custom HID device *and* a HID mouse device. Perfect for experimentation of potential HD Mouse API workflows.
NDUS wrote:
08 Nov 2020, 18:48
Re: a mouse movement API suitable for 3D applications, this sounds amazing. The sister API to WM_MOUSEMOVE that should've already been made years ago, imo.
Right on! Since you mention WM_MOUSEMOVE, obviously you know programming. Anything I missed from the laundry list requirements in this napkin-sketch proposal for a future spec?
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

NDUS
Posts: 71
Joined: 12 Aug 2019, 16:05

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by NDUS » 08 Nov 2020, 19:05

Chief Blur Buster wrote:
08 Nov 2020, 19:02
[...] Right on! Since you mention WM_MOUSEMOVE, obviously you know programming. Anything I missed from the laundry list requirements in this napkin-sketch proposal for a future spec?
Most of this is above my head, to be honest.
Only thing is I think it would not be hard to market this idea to Microsoft because they're investing a lot into marketing Windows as the platform for video games.

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 08 Nov 2020, 19:08

NDUS wrote:
08 Nov 2020, 19:05
Most of this is above my head, to be honest.
Only thing is I think it would not be hard to market this idea to Microsoft because they're investing a lot into marketing Windows as the platform for video games.
Although I can disclose that Microsoft and I have (up to this date, 2020 thus far) never had any contracts or paid projects -- I do have lots of fans at Microsoft. Brad Rosetti of XBox team confirms they are big fans) and some had said Blur Busters's advocacy made it easier to convince the team (VRR demos, etc). More Hz the merrier.

It was not easy but Blur Busters did recently convince Microsoft to raise the refresh rate limit (breaking the 500Hz barrier) in a special build of Windows which I now have installed on my desk. This feature is now available to registered Microsoft Partners / ISVs for incubating future high-Hz, though it will probably be potentially years before it hits retail builds.

I believe I now have enough pull to build some kind of open collaboration for a HD Mouse API

_________________

<Aside>
My ability to collaborate with the entire audience of gamers (from kids to elderly, from friendly researchers/scientists to being patient with university students who know less than I about high-Hz) -- no matter how little or lots they know -- has improved my reputation in a great many circles in my willingness a bigger majority many in discussions to bring more standardization.

My ability to engage with gamers, my ability to respectfully engage with good patient researchers, and suddenly suit up for management, while understanding all these high-Hz temporal concepts, is a rare combination that is not found in many researchers. I more resemble a user rather than a researcher, and I often say no to money more often than many capitalists say I should.

I cooperate with vendors that align with general Blur Busters initiatives, but I am also quick to say (to these vendors, and in legal agreements) that I am impartial / neutral and need to be allowed to work multiple vendors. For example, I wish I had more FreeSync content, but it's just a symptom of more enthusic fans in the G-SYNC community (users and vendors) than the FreeSync community, so people think I am more G-SYNC biased, but it's just a matter of fact -- I go with the flow despite asking for more people/vendors/etc to contribute FreeSync content / ideas / discussions / samples. So people see an obvious G-SYNC bias simply because of sheer happenstance. In reality, there's no agreement signed relative to G-SYNC favouritism. I love riding Hz tides lifting all boats!

Although these forums are an unconventional approach of incubation that later spawns off to official articles or later off-the-websites private collaboration, the Blur Busters Forums is my 21st century approach of scratchpading ideas on Blur Busters temporal business (much like Elon Musk & Twitter). I don't have much patience for proprietary standards too expensive/unusable by the masses. I love bridging the kids to the good patient inclusive researchers, and I have NASA-style encouraged quite a few to start studying gaming and improve programming techniques -- like how Chris Hadfield is patient with all the incessant question-asking space fans.
</Aside>
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by Chief Blur Buster » 10 Nov 2020, 14:18

Hello,

Current API Channels That Could Be Extended To HD Mouse API

While initial incubation might not be via Microsoft OS...

...Here’s Microsoft’s existing page about getting the highest definition mouse data:
Taking Advantage of High DPI Mouse Movement

However, hardware-based mouse DPI switching denies the precision being fed into even this API. In other words, we need something that accesses both the user’s preferred DPI (400-800dpi) and the full precision of the mouse sensor simultaneously. Preferably through separate APIs without the user sabotaging the DPI of their mice because a game doesn’t process high-DPI accurately.

The HD Mouse API could actually perhaps be some theoretical extension registers to WM_INPUT, possibly a theoretical extension to the same WM_INPUT event architecture, if Microsoft wanted to get involved in this.

That said, there is a lot of inertia/bureaucracy at Microsoft In getting new APIs added — so an outside API is probably needed initially (e.g. in vendor mouse drivers) that meets a standard spec, so that this all can be incubated long in advance of Microsoft including HD Mouse API into the WM_INPUT infrastructure. This is quite common (i.e. certain OpenGL driver extensions that occur before DirectX extensions)

Also, some API frameworks already use timestamped batching (e.g. arrays in HTML5 PointerEvents API), but even C/C++ benefit (e.g. HD Mouse API can get 20,000 coordinates per second at lower poll rates via 20KHz sensor readdrate over 4KHz poll at 5 transmitted reports per poll), all pre-existing batched timestamped mouse APIs can greatly improve with hardware-based batching support, since it’s a more straightfoward USB/driver/ISR copy from hardware to software. Think of it as hardware-based acceleration for high poll rates. All languages at all levels benefit with the same precision, regardless of ASM or C++ or JavaScript!

The gist of a HD Mouse API is that it should be very easy to adapt to different architectures, to meet the common goal of permanent full sensor precision access via sidestream APIs even while concurrently running existing legacy DPI-locked APIs.

Terminology Disambiguation
For the purposes of this post “CPI” and “DPI” means the same, but interpreted slightly differently from the context of HD Mouse API. Within the HD Mouse API, the “DPI” becomes the the integer pixel count for the currently user preferred “DPI” setting, excluding the new subpixel precision provided by HD Mouse API.

Behaviours when HD Mouse API is Installed/Active with Supported Mice

In RFC2119 terminology parlance (“MAY”, “SHOULD”, “MUST”, “SHOULD NOT”, “MUST NOT’, etc), a future gaming mouse supporting HD Mouse API means:
  • To the HD Mouse API, the mouse MUST provide full DPI precision at all times at the maximum non-interpolated non-smoothed precision, regardless of current DPI divisor (aka user preferred DPI).
  • Thus, a non-interpolated 12800dpi mouse will provide 800dpi (or user selected DPI) to legacy APIs and 12800dpi (or equivalent floating-point precision) to HD Mouse API — with both APIs concurrently available at all times upon plugging in the mouse.
  • Any optional smoothing/Interpolation/etc MAY be optional features of custom mouse configuration utilities in a vendor mouse driver.
  • By default, upon plugging in the mouse for the first time, there MUST NOT be any smoothing/interpolation applied to the precision data provided to HD Mouse API.
HD Mouse API minimum requirements:
  • Legacy behaviours MUST still work at their original lower user-preferred DPI, to maintain user friendly experience
    This preserves user experience familiar with legacy mice
  • Subpixel precision (integer or floating point) MUST be available via HD Mouse API.
    Subpixel ability is a prime purpose of HD Mouse API, and prevents the turbo cursor problem associated with DPI-switching
  • To avoid mouse cursors too fast for user, the middle mouse speed setting in Control Panel MUST remain consistent pointer speed at the user’s configured preferred DPI (e.g. 400dpi or 800dpi), despite full resolution (i.e. 12800dpi sensor) now being made available to newer apps via HD Mouse API.
    This is a prime purpose of HD Mouse API, provide non-turbo-speed mouse cursors yet provide subpixel precision at preferred cursor speeds
  • All mouse-coordinate/delta pairs MUST have sensor-side timestamps with an accuracy of 1 microsecond or better. This helps all programming languages from HTML5 thru C#/C/C++ thru ASM. Also, did you know? The HTML5 PointerEvents batched API already has microsecond timestamp values, so it’s easy to adapt it to use HD Mouse API data (use hardware firmware/sensor timestamps rather than jittered software timestamps).
    This resists jitter better between sensor to photons, including cable, OS, drivers, application
  • All mouse-coordinate/delta pairs MUST not be smoothed/interpolated by default, unless intentionally configured otherwise by user
    Full unmodified Sensor precision is made available
  • There MUST NOT be an assumption of a poll rate maximum. In fact, the sensor data may be pushed or interrupt-based instead of polled at the driver level (e.g. SPI, Ethernet, I2C, etc), the high level API design MUST NOT make an assumption of the underlying mouse workflow (i.e. how USB works).
    This futureproofs future gaming mouse architectures
  • There MUST be an activateable ability to batch polls (multiple accurately timestamped sensor reads in one poll), even if this ability is made optional configurable user setting
    This can greatly reduces CPU load in future supported games, as high poll Hz has load problems
  • There MUST be ability to access the full sensor read rate beyond poll rate (i.e. 20KHz) via batched polls (i.e. 20 reports per poll at 1000Hz, such as an array of 20 mouse movement delta pairs). This reduces the CPU overhead of high poll rates.
    Batching and timestamped sensor reads makes this possible and easy
  • Users SHOULD be able to configure preferences via optional advanced mouse utilities (e.g. 1 report at 8KHz, or 8 reports batched at 1KHz), for various optimizations such as reducing CPU load in newer games. While this will not happen by default, such configurability SHOULD be made available.
    Accomodate power users, while keeping things plug and play for everybody else
  • Any API we propose SHOULD be easy for manufacturers, developers and users alike, and be designed to work with the confines of existing mouse sensors & microcontrollers, while permitting innovation in new mouse-movement sensors (non-DPI-based, analog-asynchronous intervals between reports, etc)
    This is needed for HD Mouse API adoption
HD Mouse API solves a major firmware/software/UX bottleneck

Hardware parts availability is a solved problem.
Today, this is already prototypeable on any USB2-ported Arduino. They already have microsecond timestamppers, they already can do 8000 Hz poll, they can already emulate a HID mouse or a custom USB HID device of a hobbyist’s dreams. Nobody needs a big budget to test this out. The problem is simply a HD Mouse API that vendors can agree on, preferably starting simple with fewer-stakeholder involvement (i.e. no requirement for Microsoft Windows modifications initially).
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

User avatar
lyrill
Posts: 385
Joined: 06 Oct 2020, 10:37

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by lyrill » 11 Nov 2020, 04:28

don't forget eDPI. the reason people use low dpi and usually they DO indeed also mean EDPI ------ whether they realize the distinction or not ( not that they are ascended if they do it's so layman to boast about lul)------- is because it masks their suboptimal mouse skates and mouse pads which have terrible discrepancy between initiation drag coefficient and glide friction. this makes high EDPI usage basically impossible if you do micro adjustments, example, you can basically say goodbye to tracking or reaction shots of any PRECISION weaponary. Not even if you snag the mouse and try super hard to give it control when that's never supposed to be that way. Of course and this is really a huge dismay too as mouse pads that have shit initiation speed usually have pretty high glide speed (low none static friction) which means some uninitiated think it means it's a fast pad, lul. it's not. it's fast for AOE flicks, sure, for the rocket spams maybe. Having high static friction doesn't necessarily equate to having a high stopping power either. Not when it's a harder(surface) pad.

Back then everyone, and I do mean EVERYONE including the SK Reign of Terror on the CS scene, used size S to M mouse pads, that basically demand you either lift insanely often which causes jitter no matter how good the lift off performance is/how low lod is, on top of breaking up the motion control wholeness, or you just use a high dpi that's always been available ---1500/1600 dpi since 2004/2005, both laser and optical. NOBODY was misinformation spreading that either was completely garbage. Both were pushing each other tech to the next limits. It was the glory days.

And you know why it's good to have a smaller mouse pad usage habit? Because you keep your stance the same as you keep your left hand static relatively. It's like a race car that holds the driver in place with extra cupping on the seat. Nobody's break dancing in the seat while racing F1, are they? Posture matters for perf, ergonomics, and basically perf..

But yea relying on excessive sized pad goes hand in hand with relying on potato slow dpi to filter out the raw jankiness of subpar hands with subpar skills (that is to say, there is a difference in human skill, regardless of the pad/skate quality and their understanding of them)--if they were to be asked to pencil draw with an iphone 4 instead of say iphone 12 pro max sized screen but say no retina screen.---this is all a bit ironic when such demographic fear acceleration/linear correction/etc. because adding those in a reasonable way + a truly spectacular mouse pad that maybe isn't even analog (maybe the sensor just sends and let the pad receive underneath, how about that? maybe stop making corded pads just as charge pad for the mouse or for senseless rgb) would be truly ascended lol. Because let's face it, no human tracking job is perfect but with the same standard, a lot of equipment CAN be considered perfect. So yeah I am all for raw input if it actually makes sense.....smoothing is circular correction largely. And it makes more sense than having jitter fest with high dpi even if pad is high quality, just to lowering supposedly sub 5 ms when system+internet lag is usually 10 times that at least

User avatar
schizobeyondpills
Posts: 103
Joined: 06 Jun 2020, 04:00

Re: Potential HD Mouse Extensions API Specification [ETA by 2025]

Post by schizobeyondpills » 13 Nov 2020, 02:55

For API/software/driver side of things it should try to be something like io_uring is in linux

https://lwn.net/Articles/810414/
https://unixism.net/loti/what_is_io_uring.html
Because of the shared ring buffers between the kernel and user space, io_uring can be a zero-copy system. Copying bytes around becomes necessary when system calls that transfer data between kernel and user space are involved. But since the bulk of the communication in io_uring is via buffers shared between the kernel and user space, this huge performance overhead is completely avoided.
also i believe few things are a MUST such as
- adjusting affinity of mouse input usermode thread/driver
- adjusting priority of mouse input usermode thread/driver
- "idle off" mouse input thread (highest priority/can't be out-scheduled), maybe even completely isolate a CPU core for mouse only input (not even interrupts/drivers/DPCs get scheduled at that isolated mouse CPU core)
- adjusting DPC/ISR priority of mouse/driver

Post Reply