This is just a scratchpad for now;
Lots of Blur Busters stuff incubate from Laboratory area (including Area 51) which are higher % industry-lurkers (large lurk readcounts from businesses) rather than from younger fans in the General/Latency area. Over 99% of non-bot forum visitors are lurkers, and clear workday bandwidth traffic comes from Asian lurkers (i.e. the manufacturers). Eventually this will promote to an official Blur Busters page, but for now, this is just scratchpad stage. Currently inspired by this Twitter Thread between myself and Razer.
High Definition Mouse Extensions API Proposal
I've been doing this for some time, but I've finally got some things rolling that might happen by year 2025. This is my idea, not any mouse vendor's idea. Yes, I'll run with the first vendor to bite, and no, I'm not asking for money or rewards for this. And no, Razer didn't request this. But because of improving industry connections, I'm now pushing for this. It's part of my passion.
Things are now far enough along that I'm throwing a trial balloon into a piranha tank (online, offline, behind scenes, forum). This is not a real spec, this is just a discussion of a possible spec to eventually gel. The goal is something by year 2025 because we are reaching severe mouse limitations in the refresh rate race to retina refresh rates.
I have years of experience collaborating with standards authors, such as XEP-0301 In-Band Real Time Text which is used by some projects, and was adopted into some Next Generation 911 systems (so deaf can type to 911 operators, and 911 can their message typed one character at a time, lest they get interrupted/die mid-way typing) -- my XEP-0301 standard mentioned at fcc.gov. Although they ultimately adopted RFC4103 instead, it does underscore my standards experience, and the message-by-message version of RFC4103 actually borrowed some ideas later already. But I write this just to mention how beautifully formatted and RFC2119-compliant a spec document I can write.
Higher Display Refresh Rates Expose Major Limitations Of Mouse
Higher resolutions and higher refresh rates, with clearer motion (from refresh rate or strobing) amplify human-visible mouse microstutter. New 360 Hz monitors show the limitations of 1000Hz poll rate much more visibly than at 240Hz and 120Hz. Part of the reason why people in LinusTechTips YouTube video could not tell apart 240Hz versus 360Hz is because the mouse poll was only set to 1000Hz. This diminished the human-visible difference between 240Hz and 360Hz, because the difference was obscured by mouse jitter. Despite other Amazing Human Visible Feats Of The Millisecond.
Current mouse architecture hurts the market for all high-Hz displays, since the latest mouse sensors are now capable of far better precision than currently exposed to the computer/games. This is the main raison d'etre of a High Definition Mouse Extensions Specification
- 360 Hz monitors have recently showed limitations of 1000 Hz mice
- Many esports gamers want to stay at 800dpi
- High poll Hz can create undue system load
- Turbo mouse cursors occur with high DPI
- The need to be compatible with legacy games
- The need for more precision in newer games
- Different gamer preferences in DPI and sensitivity
- Low latency use cases versus smoothed use cases
- The need to stay microsecond jitter accurate in increasingly high-level game programming (C# C++ HTML5 etc)
- The need to be compatible with ultra low latency for pure C/ASM/RTOS implementations
- Current mouse architecture, derived from The Demo (Doug Engelbart), is increasingly problematic
- Becomes problematic in the light of Vicious Cycle Effect
- Other conflicts between different stakeholders (users, vendors, OS, etc)
Example 2: Another gamer sets 3200dpi (low sensitivity such as 0.06) for best Valorant performance but gets turbo mouse cursor in menus or exiting to Windows
Example 3: DPI switching manually configured by the user on a per-game basis in mouse vendor software
Example 4: Some games gets affected by the Sensitivity slider in Windows
Example 5: 8000 Hz pollrate solves some problems but creates new problems
Example 6: Gamer permamently leaves mouse at 400dpi, limiting mouseturn resolution
A new high definition mouse API that is more future proof, yet isn't too onerous for all mouse vendors and software developers.
- Backwards compatible
- Allows 400dpi and 800dpi to continue to be used
- User friendly
- No cursors too fast for users
- Way to avoid pollrate overloading CPU
- Newer games gets fuller access to mouse sensor rate
- Less frequent DPI switching
- Maintains USB HID and maintains default behaviour
- Yet newer games get full automatic access to full precision via new parallel API
- Full access to full sensor rate without poll rate granularity interference (e.g. direct 20 KHz sensor access)
- Users can stay at 800dpi yet newer games still have access to full sensor resolution
- More microsecond accurate; easier for developers to dejitter mouse
- Compatibility with connecting a mouse via a different cable more time-precise than USB (e.g. SPI, GPIO) or
- Compatibility with preserving full sensor rate even at low poll rates via poll batching with mouse-side microsecond timestamping of sensor reads
- Reduce the CPU overhead of 8000 Hz or 24000 Hz poll rates
- Maximum backwards compatibility with USB HID
- Easy for software developers who choose to support extended API
- Extended API can be turned OFF, we don't want to force this upon all users
- No medium-enforced upper limit to poll rate/sensor read rate, e.g. 20KHz sensor reads can be batched at 20 timestamped coordinates over 1000Hz USB
There are multiple potential routes of a high definition mouse extensions API, but the common goal is to get multiple mouse vendors to agree to a common extended mouse API for futureproofing the refresh rate race to retina refresh rates. An open, royalty free, patent free specification. But the route there have different pros/cons that make it easier/harder to meet the ideal. So specifications are inherently compromises. A Version 2 spec (Some ideals) that has a path to Version 3 spec (all ideals) is ideally preferred, so we don't want to architecture ourselves into a corner.
Thus, it is possible not all ideal capabilities may be included in the initial spec, however, the specification should remain extensible.
High Display Refresh Rates Need Both High DPI and High Poll Rates
High DPI fixes slowturn jitter
If you do 400dpi + 4.0 sensitivty, your slowturns are steppy which sabotages slow-movement fluidity/framerate. But you do 1600dpi + 1.0 sensitivity, slowturns are much smoother. You probably already know that. But many games are crappy at high-DPI settings, so people stay at 400dpi or 800dpi. Now even if you use 800dpi+0.5 sensitivity, remember even a half-pixel mouseturn is still human visible in some games (it looks like an antialiased mouseturn) because 3D can be rendered subpixelly. So you want subpixel precision headroom for motion nirvana, assuming the game co-operates properly with high DPI without smoothing/interpolation sheninigians.
High Poll fixes fastturn jitter
Try doing 125Hz or 500Hz mousepoll on a 240Hz monitor. It feels strangely coarse/jittery during fastturns especially when you've got blur reduction turned on (DyAc ON for example). Raise mouse to 1000Hz and the fastturn jitter is gone. Now the new monkey wrench, <Cue One More Thing Music>, raise your display refresh rate to 360Hz or higher, and make sure your framerate is in that territory. The 1000Hz fastturn jitter faintly comes back in some situations in some games (via watching the stroboscopic effect in-game with your eye on high-contrast edges in-game). Ouch. So the display Hz is now too close to mouse poll Hz and no longer human-invisible. Time to punt up the mouse pollrate a smidge to max out human benefit at least in the visibility department. 2000Hz at least, to avoid the overload factor.
Subpixel Mouse Turns Are Human Visible
Even a 0.5 pixel mouse turn can be human visible, it just looks like an antialiased mouse turn, since 3D graphics can be rendered at subpixel levels. Valorant does this well, while CS:GO is not as good at this. That's why Valorant works well at higher DPI than CS:GO does (in other words in Valorant, 3200dpi 0.06 sensitivity feels identical to 400dpi 0.48 sensitivity, for fast flick turns, on a mouse without smoothing for 3200dpi. Except now slowturns now benefit).
Need To Stay 400dpi and 800dpi Compatible For Legacy
Many legacy games such as CS:GO do not behave very well at 3200dpi. Even when the sensor has smoothing/interpolation/etc disabled, 3200dpi at 1/4th sensitivity you use at 800dpi -- don't necessarily behave the same because the old engine didn't scale well for high-DPI operation.
Easier Preservation of Math Precision for Mouse Coordinate Mathematics
I even now recommend software developers to cast (int) mouse coordinates to (double) instead of (float), as a higher resolution intermediate processing step (explained in HOWTO: Please Futureproof Your Game Engine). Much like the 10bit color processing step from an 8bit GPU to an 8bit panel, is still worthwhile (less color rounding errors). But that's a programming step that developers have to actively remember to do. High Definition Mouse Extensions API will help automate the preservation of precision for software developers.
Long Term Benefits to Windows/Linux/Mac Scrolling
Full DPI information for scrollbar scrolling is beneficial. For example, you load a tall webpage/document/code, and try to drag a scrollbar by 1 pixel? The webpage scrolls by multiple pixels at a time. Ouch. But if you could access 12800dpi information while staying at 800dpi mouse pointer speed, the scrollbar would gain 16 subpixel steps per 1-pixel scrollbar movement. Drag-scrolling becomes much smoother and more pleasant, especially on a high-Hz display -- reading smooth scrolling text on a CRT-motion-clarity display makes finding text during scrolling faster.
Scrollbar widgets supporting HD Mouse API would massively smooth scrolling for software that implemented the HD Mouse API compatible scrollbars -- tall texts would scroll much more smoothly on high-Hz displays.
Biggest benefit will be games; that said HD Mouse API benefits UX more than just games.
Possible Proposals For Mouse Transmission
These are just musings of possible routes to achieve the ideals, with respective pros/cons. This is the behind-the-scenes communication channel, not what is visible to games.
1. Mouse Side: Pure HID Protocol Only
1. Mouse sends full resolution coordinates in existing HID protocol (e.g. 12800dpi without smoothing/interpolation)
2. Mouse driver would handle the divisors (e.g. divide 12800dpi numbers by 16 to get 800dpi numbers)
3. All existing mouse APIs stay legacy compatible, with same old fashioned DPI switching behaviours
4. The magic is that a new HD Mouse API becomes available to newer games, accessing the full original DPI
Pros: Works out-of-box with Windows 10 mouse driver. Less involvement from OS/Microsoft since it fits better in bog standard USB HID.
Cons: Less extensible to meet various specific ideals (some can be done via HID sidechannels, like those used to signal changes to DPI/poll/etc). More research on sidechannel abilities is needed, to see what can be done within the confines of the standard Windows 10 mouse driver (e.g. hidden extended information attached to each mouse poll, which is already being done, this could be piggybacked upon -- but more research is needed)
2. Mouse Side: Custom Protocol
1. Mouse concurrently sends standard USB standard HID packets and high-definition packets
2. Without installing mouse drivers, Microsoft only sees the USB standard HID packets
3. With installing mouse drivers, the extended packets containing extended information can be processed.
4. The extended packets would only be handled by newer games that accesses the HD Mouse API
It is also theoretically possible two separate USB devices appear when plugging in the mouse, but the hub approach needs to be avoided where possible, due to latencies.
Pros: Much more extensible, such as poll-batching and poll timestamping. For example sending an array of multiple separate mouse-firmwareside-microsecond-timestamped sensor reads in one single USB packet, allowing 20KHz sensor via 1000Hz/2000Hz/4000Hz/8000Hz poll even at user configurable thresholds that accomodates user preferences of lag/precision/etc. Extended packet format can be formatted in a way in a USB HID sidechannel compatible way *and* non-USB methods (e.g. SPI, GPIO, Thunderbolt alt modes, etc)
Cons: Massive increases in complexity of hardware engineering, due to protocol, but much more extensible.
Possible Proposals For Software Processing
This is what becomes visible to software that wants to access the new HD Mouse API.
- The HD Mouse API would have metrics to inform the game (sensor rate, poll rate, DPI, etc)
- The HD Mouse API would provide full precision information either as integers or as floating point
- The HD Mouse API provides optional poll batching to reduce system load
- The HD Mouse API can fire additional events for special events (DPI changed, mouse lifted from surface, etc)
- The HD Mouse API can have API-accessible configurability such as poll batching, smoothing ON/OFF, smoothing-vs-lag windows,
- The HD Mouse API will include floating point timestamps (double) or monotonically increasing timestamps (no less accurate than 1 microsecond accurate) for each coordinate pair and button events, with timestamps recorded at sensor level.
- The HD Mouse API will inform application where the timestamps came from (sensor side versus driver side), to allow applications to trust the integrity of the timestamps.
- The game would handle the data as it wishes, ideally with:
......Separate sensitivity adjustment for mouse pointer for menus/inventory/etc
......Separate sensitivity adjustment for mouse turns
......Separate sensitivity adjustment for scopes
Sensitivity adjustment MUST (RFC2119) have at least 4 decimal digits, as explained at HOWTO: Please Future Proof your Game Engine
Optional poll batching reduces system load. To gain sensor rate higher than USB rate (e.g. 20KHz "poll" via 2KHz USB poll via a single USB packet transmitting an array of 10 sensor reads and accompanying timestamps recoreded at sensor level), to allow newer games to access sensor read rates without the overload associated of high poll rates. The poll rate controls the latency you want but you can't just always reliably have jitter-free higher poll rate on some systems. Poll batching should be optional (i.e. a checkbox)
Also, all this can be combined independently with above possible proposals for mouse transmission (e.g. 1+A, 1+B, 2+A, or 2+B). This is only for the HD Mouse API only accessible to newer games. This does not affect legacy games which would continue to get the old-fashioned ordinary behaviors (behaving identically as before).
A. Integer Mouse Coordinates
Full precision numbers would be provided, i.e. if the mouse is 12800dpi, the numbers provided will be always 12800dpi (example) in the HD Mouse API even while the legacy mouse APIs concurrently gets 400dpi or 800dpi numbers (at user dpi preference, as usual). The game will just divide as necessary via high-precision sensitivity adjustments. In other words, permanent new-game access to full mouse precision becomes available if user enables the HD Mouse API.
- Simpler for engineering mouse
- Requires more work/effort by developers
- Turbo mouse cursor may happen more often by accident, user unfriendly
B. Floating Point Mouse Coordinates
*** Blur Busters Preferred Route ***
The mouse drivers would divide the integer data by the appropriate divisors in the mouse driver before exposing them in the HD Mouse API, without rounding off. Users would configure their preferred DPI as usual, like 400dpi, 800dpi, like always. The bonus is on a 12800dpi mouse (non-interpolated/non-smoothed) "400dpi" through "12800dpi" would preserve same precision thanks to the decimal digits. No legacy wonky mousefeel. It would be integer coordinates for the legacy API, and floating point coordinates for the HD Mouse Extensions API -- identical when rounded to the nearest integer.
- More user friendly. Avoids the turbo cursor effect. You're getting "800dpi" numbers with 12800dpi (or any DPI) precision
- Infinitely upgradeable. You can get "400dpi" or "800dpi" from any DPI (such as 12800dpi) with same precision of max DPI
- Fewer separate sensitivity adjustments needed for users; same setting for mouse turns and mouse pointer is easier.
- More flexible to future nontraditional sensor workflows (true asynchronous sampling, vector sensors, etc)
- API will return (double) coordinate deltas to encourage good developer habit to keep higher precision intermediate math processing to reduce rounding errors.
- Minor amount of game developer education to explain benefit of floating point mouse coordinates
Alternative ideas that are feasible to manufacturers
There are other possible avenues, but the important thing is that the specification needs to remain relatively simple. Complicating the specification more means more involvement by more vendors (e.g. OS/Microsoft, USB chipset makers, etc) so it has to be designed in a way that minimizes changes to the mouse chain (while adding mouse firmwareside microsecond timestamping of polls + adding optional batching support) and stay easy for users. A user needs to be able to plug in a mouse, and it just automagically works.
*** Please note, this is an early pre-specification napkin proposal of a discussion ***