VRR for video playback (video players, web browsers, etc.)
Posted: 06 Dec 2024, 04:30
I finally got a VRR monitor for myself and I'm loving it. I'm a person who would always be fiddling with monitor's refresh rate to match the fps of whatever I was doing (gaming, video playback, etc.) and finally I don't need to do that for games anymore. However, for video playback that is not as straight forward, especially on the web browser and/or streaming side of things. Native video players exist that support VRR, others support automatical refresh rate change based on the video's fps, but no web browser / streaming app that I know of support VRR.
Sadly I am not a developer, but I have prepared argumentation of why web browsers and streaming service apps should consider supporting VRR:
Advantages of VRR for Video Playback
Handling Mismatched Frame Rates:
Many modern monitors have high refresh rates (e.g., 144Hz), which work well for frame rates like 24fps, 48fps, or 72fps but cause issues with non-matching rates like 30fps or 60fps due to uneven frame delivery.
With VRR, the display's refresh rate can dynamically match the frame rate of the content or its multiples (e.g., doubling 24fps to 48fps to avoid flicker and maintain compatibility).
Avoiding Display Judder:
Non-integer frame rate matching often leads to judder, where frames are repeated inconsistently. VRR eliminates the need for such compromises by syncing frame presentation directly to the monitor.
Web Browser and Player Integration:
Web-based video playback (e.g., YouTube) often operates in environments where the refresh rate is fixed at the monitor’s maximum. Enabling VRR would allow browsers to adjust refresh dynamically, offering smooth playback for diverse content. I am watching YouTube videos that are sometimes 24, 25, 30, 50, 60 fps and without VRR I would need a 600 Hz refresh rate to display them all smoothly, with VRR no such high refresh rate would be needed.
Challenges and Workarounds
Low Frame Rate Limitations in VRR:
Many VRR monitors do not support very low frame rates (e.g., 24fps). A good workaround is to use frame rate doubling or tripling, e.g., presenting 24fps video at 48Hz or 72Hz within the VRR range.
Players should implement logic to always match at least a double or triple multiple of the source frame rate.
Audio Sync:
Adjusting the refresh rate dynamically may cause timing mismatches with audio playback. This requires careful synchronization mechanisms in the player. However, this might not be a real issue at all, since video fps is suppsed to be constant.
Web Browser and System Integration:
Browsers and operating systems must expose APIs for VRR control.
Limited Standardization:
Different monitors and GPU drivers handle VRR slightly differently, making a one-size-fits-all solution difficult. Developing a robust VRR implementation would need careful testing across a range of hardware.
Implementation Proposal
Web Browser Support:
Browsers like Chrome, Firefox, and Edge could expose VRR controls to web apps, allowing YouTube and other streaming services to utilize VRR for smoother playback.
System APIs for VRR:
Platforms like Windows, macOS, and Linux should allow finer VRR control for media playback applications, enabling smooth transitions between different refresh rates.
Monitor-Specific Optimizations:
Monitor manufacturers could expand VRR ranges downward (e.g., below 30Hz) or implement features like frame-doubling internally to support smooth playback of low frame rate content.
Potential Projects and Future Work
Custom VRR Video Playback Pipelines:
Developers could create web browser extensions and open-source plugins or patches for existing players like MPV or VLC to experiment with VRR for video playback.
VRR Middleware:
Middleware software could act as a bridge between the player and display, dynamically adjusting refresh rates as needed.
Collaborations with Hardware Vendors:
Teams working on VRR video playback could collaborate with GPU and monitor manufacturers to optimize the technology specifically for media playback scenarios.
Suggestion to set VRR to a double or multiple of the frame rate is particularly compelling, as it ensures smooth playback even on monitors with limited VRR ranges. This approach would make VRR adoption for video playback not only feasible but also highly practical for both desktop applications and streaming platforms.
So, in conclusion... I would love to see the same attention that gaming on VRR has paid to video playback on VRR so we can forget fiddling with monitor's refresh rates for an always smooth video playback, no matter the fps of the video.
Sadly I am not a developer, but I have prepared argumentation of why web browsers and streaming service apps should consider supporting VRR:
Advantages of VRR for Video Playback
Handling Mismatched Frame Rates:
Many modern monitors have high refresh rates (e.g., 144Hz), which work well for frame rates like 24fps, 48fps, or 72fps but cause issues with non-matching rates like 30fps or 60fps due to uneven frame delivery.
With VRR, the display's refresh rate can dynamically match the frame rate of the content or its multiples (e.g., doubling 24fps to 48fps to avoid flicker and maintain compatibility).
Avoiding Display Judder:
Non-integer frame rate matching often leads to judder, where frames are repeated inconsistently. VRR eliminates the need for such compromises by syncing frame presentation directly to the monitor.
Web Browser and Player Integration:
Web-based video playback (e.g., YouTube) often operates in environments where the refresh rate is fixed at the monitor’s maximum. Enabling VRR would allow browsers to adjust refresh dynamically, offering smooth playback for diverse content. I am watching YouTube videos that are sometimes 24, 25, 30, 50, 60 fps and without VRR I would need a 600 Hz refresh rate to display them all smoothly, with VRR no such high refresh rate would be needed.
Challenges and Workarounds
Low Frame Rate Limitations in VRR:
Many VRR monitors do not support very low frame rates (e.g., 24fps). A good workaround is to use frame rate doubling or tripling, e.g., presenting 24fps video at 48Hz or 72Hz within the VRR range.
Players should implement logic to always match at least a double or triple multiple of the source frame rate.
Audio Sync:
Adjusting the refresh rate dynamically may cause timing mismatches with audio playback. This requires careful synchronization mechanisms in the player. However, this might not be a real issue at all, since video fps is suppsed to be constant.
Web Browser and System Integration:
Browsers and operating systems must expose APIs for VRR control.
Limited Standardization:
Different monitors and GPU drivers handle VRR slightly differently, making a one-size-fits-all solution difficult. Developing a robust VRR implementation would need careful testing across a range of hardware.
Implementation Proposal
Web Browser Support:
Browsers like Chrome, Firefox, and Edge could expose VRR controls to web apps, allowing YouTube and other streaming services to utilize VRR for smoother playback.
System APIs for VRR:
Platforms like Windows, macOS, and Linux should allow finer VRR control for media playback applications, enabling smooth transitions between different refresh rates.
Monitor-Specific Optimizations:
Monitor manufacturers could expand VRR ranges downward (e.g., below 30Hz) or implement features like frame-doubling internally to support smooth playback of low frame rate content.
Potential Projects and Future Work
Custom VRR Video Playback Pipelines:
Developers could create web browser extensions and open-source plugins or patches for existing players like MPV or VLC to experiment with VRR for video playback.
VRR Middleware:
Middleware software could act as a bridge between the player and display, dynamically adjusting refresh rates as needed.
Collaborations with Hardware Vendors:
Teams working on VRR video playback could collaborate with GPU and monitor manufacturers to optimize the technology specifically for media playback scenarios.
Suggestion to set VRR to a double or multiple of the frame rate is particularly compelling, as it ensures smooth playback even on monitors with limited VRR ranges. This approach would make VRR adoption for video playback not only feasible but also highly practical for both desktop applications and streaming platforms.
So, in conclusion... I would love to see the same attention that gaming on VRR has paid to video playback on VRR so we can forget fiddling with monitor's refresh rates for an always smooth video playback, no matter the fps of the video.