High Refresh Rate Rendering Pipelines

Talk to software developers and aspiring geeks. Programming tips. Improve motion fluidity. Reduce input lag. Come Present() yourself!
Post Reply
User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

High Refresh Rate Rendering Pipelines

Post by Chief Blur Buster » 17 May 2020, 13:55

Although this is Android related, the graphs are very relevant to various workflows (game rendering, windows compositing):

High Refresh Rate Rendering on Android

I'm crossposting some beautiful graphs that helps one visualize a high refresh rate rendering workflow, with stutter-reduction algorithms:
Google Blog wrote:Rendering at high rates

The higher the rendering rate, the harder it is to sustain that frame rate, simply because there is less time available for the same amount of work. To render at 90Hz, applications only have 11.1ms to produce a frame as opposed to 16.6ms at 60Hz.

To demonstrate that, let’s take a look at the Android UI rendering pipeline. We can break frame rendering into roughly five pipeline stages:
  1. Application’s UI thread processes input events, calls app’s callbacks, and updates the View hierarchy’s list of recorded drawing commands
  2. Application’s RenderThread issues the recorded commands to the GPU
  3. GPU draws the frame
  4. SurfaceFlinger, which is the system service in charge of displaying the different application windows on the screen, composes the screen -and submits the frame to the display HAL
  5. Display presents the frame
The entire pipeline is controlled by the Android Choreographer. The Choreographer is based on the display vertical synchronization (vsync) events, which indicate the time the display starts to scanout the image and update the display pixels. The Choreographer is based on the vsync events but has different wakeup offsets for the application and for SurfaceFlinger. The diagram below illustrates the pipeline on a Pixel 4 device running at 60Hz, where the application is woken up 2ms after the vsync event and SurfaceFlinger is woken up 6ms after the vsync event. This gives 20ms for an app to produce a frame, and 10ms for SurfaceFlinger to compose the screen.

Diagram that illustrates the pipeline on a Pixel 4 device
Image

When running at 90Hz, the application is still woken up 2ms after the vsync event. However, SurfaceFlinger is woken up 1ms after the vsync event to have the same 10ms for composing the screen. The app, on the other hand, has just 10ms to render a frame, which is very short.

Diagram of running on a device at 90Hz
Image

To mitigate that, the UI subsystem in Android is using “render ahead” (which delays a frame presentation while starting it at the same time) to deepen the pipeline and postpone frame presentation by one vsync. This gives the app 21ms to produce a frame, while keeping the throughput at 90Hz.

Diagram app 21ms to produce a frame
Image

Some applications, including most games, have their own custom rendering pipelines. These pipelines might have more or fewer stages, depending on what they are trying to accomplish. In general, as the pipeline becomes deeper, more stages could be performed in parallel, which increases the overall throughput. On the other hand, this can increase the latency of a single frame (the latency will be number_of_pipeline_stages x longest_pipeline_stage). This tradeoff needs to be considered carefully.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

diakou
Posts: 83
Joined: 09 Aug 2020, 11:28

Re: High Refresh Rate Rendering Pipelines

Post by diakou » 09 Aug 2020, 11:30

How would this behave and/or matter for a game inherently locked at 60fps on an android or iOS device running at higher refresh rates?

Is it similar to the concept on PC/windows based systems?

User avatar
Chief Blur Buster
Site Admin
Posts: 11647
Joined: 05 Dec 2013, 15:44
Location: Toronto / Hamilton, Ontario, Canada
Contact:

Re: High Refresh Rate Rendering Pipelines

Post by Chief Blur Buster » 11 Aug 2020, 11:53

diakou wrote:
09 Aug 2020, 11:30
How would this behave and/or matter for a game inherently locked at 60fps on an android or iOS device running at higher refresh rates?

Is it similar to the concept on PC/windows based systems?
Yes, generally, anything with standard framebuffer 3D rendering (OpenGL, Metal, DirectX, etc), with a discrete refresh cycle, will tend to have all the same types of framebuffer pipelines. A specific framerate (i.e. 60fps) on a specific Hz (i.e. 120Hz) fixed-Hz on a specific sync technology (i.e. VSYNC ON) tend to have similar looking framebuffer gantt charts/timegraphs on all OS platforms. The queue depth may vary (software-side, driver-side, and OS-side) but will otherwise be similar.

This applies when a frame rate exists, a refresh rate exists, and a frame buffer architecture exists.
Head of Blur Busters - BlurBusters.com | TestUFO.com | Follow @BlurBusters on Twitter

Image
Forum Rules wrote:  1. Rule #1: Be Nice. This is published forum rule #1. Even To Newbies & People You Disagree With!
  2. Please report rule violations If you see a post that violates forum rules, then report the post.
  3. ALWAYS respect indie testers here. See how indies are bootstrapping Blur Busters research!

Post Reply