How does a DP sink reconstructs the frame

[CB_cat]

Background:

DisplayPort has 4 link rates at which video data transmission occurs. Link clocks associated with these link rates are usually faster than the pixel clock of a desired resolution. Pixel clock is the rate of transmission of pixels per unit time.

To compensate this difference between pixel and link clocks, a concept of transfer unit is used which tells us that out of x number of bytes, how many bytes will be having the pixel data and the rest of them will be fill data.

Now the value of these byte calculation at many times comes out to be a fractional value, and because of this, the bytes sent per TU are +-1 than the actual value and this deviates the overall video timing requirements.

How do I know what is the timing error tolerance in a DP sink (a monitor), for which it can construct the proper image?
And I'd also like to know how does a DP sink reconstructs the image as there is very less info on this in the DP spec.

I went through a detailed blog on this: http://billauer.co.il/blog/2015/07/displayport-tu-msa/ and it seems that the concept of TU can be ignored. I'm not sure though.

I tried searching about the working of display controller ICs but could not get good info on this. If you know any such IC which has good documentation on this topic, please guide me towards it.

[Chief Blur Buster]

How do I know what is the timing error tolerance in a DP sink (a monitor), for which it can construct the proper image?
And I'd also like to know how does a DP sink reconstructs the image as there is very less info on this in the DP spec.

It is best not to assume what a DP sink does.

Some buffer them for a whole refresh.
Some stream them straight to the display pixels.

The important thing is you need to stay in spec, tick tock, keep the metronome rocking, spew the pixel rows at a constant scan rate, and let the sink do its job.

The intervals are vestigals from an analog era but still serves as equivalent of guard interval paddings and next-item separators, helpful for early slow digital transceivers and nearly-bufferless 1:1 digital to analog adaptors (e.g. HDMI to VGA signal conversion)

Some things in HDMI and DisplayPort are somewhat ambigious, but it helps to understand the history of HDMI & DisplayPort as being aspirational temporal 1:1 versions of old analog signals.

That's why relatively passive adaptors were possible for converting analog to digital (no frame buffer), and even in today's micropacketized HDMI/DisplayPort, adaptors have memory less than one refresh cycle's worth of frame buffer.

Some DP sinks buffers the whole image and delivers it to the screen only upon full image.

Other DP sinks will stream the pixel rows straight to the screen after pixel-row processing (color processing, overdrive processing, etc) -- at the horizontal scan rate, with a small tape-delay of a few scanlines between signal's scanout position and panel's scanout position. Many esports LCDs can refresh on the fly, with signal scanout streamed more directly to panel scanout -- you can see that LCDs generally refresh top to bottom at in high speed videos www.blurbusters.com/scanout --

Modern Esports digital displays achieve subrefresh latency (like a CRT tube can) by doing near instant signal-to-screen behavior -- refreshing the screen in realtime while still receiving pixels -- getting very close to CRT latency by using subrefresh tiny rolling-window buffers of only a few scanlines (to accomodate tiny timing errors from things like DisplayPort packetization and demultiplex of other packets such as audio, and to accomodate most common processing digital algorithms such as bicubic scaling / color adjustments / etc).

Now, the scaler/TCON will tick-tock at an exact horizontal scan rate, they won't have any timing jitter tolerance. If the rolling-window buffer empties early, the screen will glitch immediately and go out-of-sync. So you don't want to deviate away from the horizontal scanrate much at all -- deliver those pixel rows as quickly as possible from the DP source to DP sink, at the most accurate timing possible, preferably exceeding spec whenever possible.

Due to this, you don't want to be tardy with delivering pixels to your DP sink, or it will go out of sync. Esports displays are very demanding that you don't lag by more than a few pixel rows, so you need to push the pixels to the sink as quickly as possible, just like you'd do one analog pixel row at a time to a CRT electron gun.

To keep input lag down, and to allow esports display to real-time stream from DP/HDMI straight to the screen (only a few pixel rows latency), with only rolling-window processing logic, you really need to keep delivering the pixel rows within very tight timing tolerances; some displays will be far more picky than others in how much DisplayPort jitter there can be.

Going beyond spec is not recommended, given the existence of ultra-low-lag tiny-rolling-window-processing logic when scanning out from cable to panel in a 1:1 scanout velocity manner (like most esports LCDs do now).

It's rather interesting how we're still delivering and refreshing in the same way between a 1930s television set and a 2020s HDMI/DisplayPort, in that we've preserved the analog-era porches and sync intervals.

[CB_cat]

Thank You for a very detailed answer.

This confirms my assumption that I have to closely follow the line timings, not much room for deviation there.

If you have any resource suggestions for the following terms/questions from your post, please share it:

• What is an Esports digital display?

• TCON -> Timing controllers; I have not found any comprehensive datasheets for DP sink ICs and those I've found are not much helpful.

• It would also be great if you could expand on Pixel clock and the way it is being conveyed in Mvid vs constant Nvid, And does asynchronous clock mode in the MSA means varying values of Mvid be used for variable refresh rates ie VESA adaptive sync?

The DP spec goes into great details for a DP source, but doesn't provide enough insights on how the DP sink will be interacting with the
Display Controller ICs.

[Chief Blur Buster]

• What is an Esports digital display?

Sorry, it’s a generic catch all term for “LCD displays that are 144Hz or higher”.

In competitive gaming, where people earn money in careers, even the olympics milliseconds-matters first when two competitors see each other at the same time, react at the same time, the display with a millisecond less lag could win more. You might not notice the millisecond unless you see the scoreboard, but these milliseconds matters immensely on these esports LCDs.

From an engineer’s perspective, these modern Esports LCDs (aka LCDs that are 144Hz and higher) are marvels of low-latency optimization and low-latency engineering. This probably isn’t an issue for you, and you can afford to do simpler programming with 1 frame buffering rather than complex code that streams the pixels from cable to panel as quickly as possible. Seeing computer software turn into photons in as little as 2-3milliseconds in certain cases (including DisplayPort overheads and LCD GtG overheads), is pretty impressive, as websites such as RTINGS use their software-based lag testers for Present()-to-photons. That’s some incredible attention to achieve sub-refresh latencies along the whole chain from application to photons.

For you, you don’t have to worry about this, just keep it simple. Just be mindful that there are many workflows for a DP sink and a TCON.

• TCON -> Timing controllers; I have not found any comprehensive datasheets for DP sink ICs and those I've found are not much helpful.

There’s an infinite number of DP sink towards TCON workflows.

The chain between the sink and TCON can do a lot.

It can buffers and split the refresh cycle into multiple temporally-dithered refresh cycles, such as plasma displays and DLP projectors.

It can buffers one refresh cycle and does its own processing filters/stuff on it (Sharpen, Brighten, Contrast, Interpolate to 120fps, etc) and scans it out sequentially (top to bottom).

It can just only line-buffers one (or few) pixel rows at a time, and refreshes the display in realtime.

It can various kinds of scan-conversion (e.g. converting 60Hz to 240Hz) as a part of an interpolation algorithm.

It can compress to H.265 and record directly to disk (e.g. HDMI video recorders that doesn’t have a display) instead of displaying it.

It can just convert the bitstream to another compatible format (e.g. DisplayPort-to-HDMI, DisplayPort-to-VGA).

Whatever you want to do between the DisplayPort end and the TCON end, is up to you.

Sometimes, some logic is sometimes built into the TCON (e.g. line buffers) so you just have a tight loop to copy DP to TCON respecting timings. Other TCONs are relatively dumb and you need to do more.

Many modern consumer displays also have advanced technologies (Variable refresh rate is simply a variable-size Back Porch, a dynamic blanking interval). They can be literally universal multisync display that can switch refresh rates instantly with zero black-out. Where the refresh rate can change 100 times a second to perfectly match a video game’s dynamic frame rate. You probably don’t need to worry about this, but it’s a clever piggyback of a minor modification of an old signal topology.

If you’re just doing a very simple display with just one mode, and no picture adjustments, then your programming between the DP sink and the TCON is probably relatively simple.

Remember that things like gamma correction (converting the DP’s nonlinear luma color space to the display’s linear color space) has to be done in your logic, but I bet you already knew that (you do know the gamma-correction math formulas, or are using a chip that does it automatically, I presume)

Some TCONs and scalers are combination units too, e.g. a single FPGA processing the signal AND driving the panel.

So there are infinite workflows, and thus outside the scope of your documents.

I dont’ have specific recommendations, because this is so broad, and simple displays just use simple off-the-shelf chips to do the picture processing. You will have to research the parts necessary, specific to your custom application (e.g. gaming display, kiosk display, office display, etc). Some can’t be shared due to NDA, while others are just really bog-standard stuff (eg. ATSC 1080p chips). Also, legacy displays (e.g. NTSC) may require obtaining NOS parts (New Old Stock) due to their discontinuance. I have no idea about NTSC parts, so I can’t help you there, but you will have to research these.

• It would also be great if you could expand on Pixel clock and the way it is being conveyed in Mvid vs constant Nvid, And does asynchronous clock mode in the MSA means varying values of Mvid be used for variable refresh rates ie VESA adaptive sync?

Pixel Clock never varies during VRR.

VRR done is done at:
- Fixed Pixel Clock
- Fixed horizontal scan rate
- Fixed horizontal intervals
- Variable vertical intervals (Back Porch varies)

If you were born before the 1980s and used CRT tubes, and remember seeing the black bar during VHOLD picture rolling -- imagine VRR as a variable-height VHOLD black bar. That’s just what it is — extra scanlines in VBI to adjust temporal spacing between refresh cycles.

The DP spec goes into great details for a DP source, but doesn't provide enough insights on how the DP sink will be interacting with the
Display Controller ICs.

There are infinite workflows. See above.

[CB_cat]

Thank you again for getting into great details. Blur Busters is probably the only go to place for getting insights like this in the field of image/video processing. Much appreciated.

[Chief Blur Buster]

Thank you again for getting into great details. Blur Busters is probably the only go to place for getting insights like this in the field of image/video processing. Much appreciated.

Thanks for the compliment. We're actually not the only place -- we're just one of the few that understands Present()-to-Photons as an entire chain; so we're a favorite when we need to communicate a "Big Picture".

In a mature market of industry standards, the problem is the supply chain is so spread thin, that everyone only understands one nut and bolt at a time.

- The DisplayPort chip maker.
- The scaler chip maker or the FPGA programmer. (DP Sink knowledge goes here)
- The TCON chip maker.

They often don't specialize in what each other does. They just sing off the same songsheet (industry standards) of passing the batons (pixels) to the next step. So asking TCON questions to the DisplayPort chip maker is often non-starter, because of the astounding number of different workflows down a DP sink.

If no chip exists for what you do, or the chip is out of stock, you can usually recreate it in an FPGA. I don't know FPGA programming but I have worked with a lot of displays that used FPGA's as the "combination DP sink / scaler / TCON". So clearly, it's pretty common, especially with new displays (e.g. first 240Hz gaming display, since nobody had released off-the-shelf 240Hz parts yet).

Sometimes the vendor selling the display simply doesn't know everything inside it, because they just outsourced the design, or it's a very generic Alibaba design, modified to suit, etc. Sometimes you have off-the-shelf panels that has pretty obscure scaler/TCONs from somebody else, and they're just reselling a generic 60Hz display. And then just have to follow the DisplayPort spec to make the picture appear, and ending up just calling it a day like most.

The advantage of the industry standards for separate portions, means interchangeability.
The disadvantage is that sometimes you run into situation of generic displays with poor instructions on how to deliver the DisplayPort pixels.
- Your displayport chip docs stays skimpy about anything beyond its sink demarcation line
- Your scaler chip docs stays skimpy about anything that a HDMI/DP chip does
- Your LCD panel docs stays skimpy about anything before the scaler.

There's some overlap, but you have to rely on the whole vendor chain to connect the dots. If you're missing a step (e.g. a display panel with NO displayport sink), then you've got to add parts such as a supported scaler until it all overlaps. So sometimes it's a hit-and-miss of confusing research. The workflow of the DisplayPort towards the display can vary by a huge amount, because not all displays refresh in exactly the same way.

And with careful display selection (if targetting simplicity over customization) -- sometimes things become simpler (e.g. an LCD panel with a scaler that has built-in Embedded DisplayPort circuit traces), and then you just have to literally connect the dots via real circuit paths, few intermediate electronics components. We don't specialize in that stuff, but we can tell you that many raw panels can either have an eDP (Embedded DisplayPort) built in, or doesn't have an eDP built in. A multi-input display will usually use an intermediate scaler that handles multiple DP/HDMI/USB-C (DP Alt Mode), but if you just need one input, then panels with built-in eDP may simplify. Depending on your use case.

That being said...

If you would like paid services from Blur Busters (services.blurbusters.com), please contact [email protected] -- to see if it's our ball or at least we can figure out any specific referrals (e.g. NDA stuff). We work with multiple display manufacturers.