I've found myself posting a lot about latency, LCD, CRT, upscaling, and recording details and pretty much repeating myself. This is from my write-up on reddit
Let's start with a flow of data.
Data Flow
Console > Cable > Display
or
Console Graphics Card > Output Chip > Console Output Connectors > Cables > Display Input Connector > Input Chip > Screen
That's generally how it flows and I'll try to cover each.
To clarify, the term Input Lag means the time it takes for what you do on your input device (your GameCube controller) to appear on the screen. Sometimes it's defined as the time it takes a device to display what's being received from the connectors, but it generally should be everything together. (Wavebirds, converters, deinterlacers, upscalers, etc).
As for input lag, the reality is, complains about input lag has near nothing to do with reaction time. No one is going to react to something an input something a frame later. The brain doesn't really work that way the average response time is something about 100ms while a good response time is 30-50ms (world record is 10ms).
Latency when playing really only matters to timed responses like powershielding/perfectshielding an attack or Smash DI input. With a 1 frame lag on a display one could arguably state that input window is shortened by 1 frames. (In reality, it's offset by 1 frames).
Adapters/Converters
The following is a list of converters/adapters with highly negligible lag (nanoseconds)
(Source) - (Destination)
Composite - Composite, SVideo, Component
SVideo - Composite, SVideo, Component
Component - Composite¹, SVideo¹, Component², VGA³
VGA - Component³, VGA
DVI - DVI, HDMI
HDMI - DVI, HDMI
¹ Format must be 480i
² Format cannot be changed
³ Format must be 480p, 720p or 1080p
Any conversion from an analog signal (Component) to a digital signal (HDMI) will incur Analog to Digital Conversion (ADC) lag.
Any conversion from an Interlaced (480i) format to a Progress Scan format (480p) will incur deinterlacing lag.
Any conversion from a higher resolution (1080p) to a lower resolution (480p) will incur downscaling lag.
Any conversion from a lower resolution (1080p) to a higher resolution (480p) will incur upscaling lag. Depending on the upscale process, this may be negligible.
FAQs
Let's start with a flow of data.
Data Flow
Console > Cable > Display
or
Console Graphics Card > Output Chip > Console Output Connectors > Cables > Display Input Connector > Input Chip > Screen
That's generally how it flows and I'll try to cover each.
- Console Graphics Card
All graphics cards compose and render data into digital RGB (Red, Green, Blue) colorspace encoding.
The game will render to a target resolution represented in pixels.
The GameCube or Wii will render your game at roughly 640x480. That's 640 pixels wide. 480 pixels high.
The graphics card will render 60 full frames per second.
GameCube and Wii render in 480p.
The Wii U will render Wii U games at 480p, 720p and 1080p. It will render Wii games at 480p. It's rumored hacked GameCube images can render up to 1080p
At this point, we could consider this as digital video data. - Output Chip
The graphics card will pipe this data over to an output chip. All consoles have digital to analog converter chips for analog output.
The GameCube and Wii U support digital output.On the Gamecube, this data simultaneously is piped to two connectors in different formats. One for the Digital AV Port and another one for the Analog AV port.
The Wii, obviously only has one port and this port is analog.
The Wii U has both an analog port and an HDMI port which is digital.- Analog
So the video data from the graphics card is piped to this chip and converted. This process is known as Digital to Analog Conversion (DAC).
Analog data can be presented in different formats and vary per console:- GameCube - 480i only
- Wii - 480i and 480p
- Wii U - 480i, 480p, 720p, 1080i, and 1080p
The Wii U's 1080p over analog is poor quality due to the limitations of the analog connection
Converting digital to analog has a bit of latency attached to it. As far as I know, no one has tested this but I wouldn't be surprised if it was less than a millisecond.
Analog signals are almost always used for 480i output. - Digital
Because the graphics card is digital and the output is meant to be digital, very little needs to be done to prepare a digital signal for output.
The GameCube will output 480i and 480p over the digital port. This RGB data is prepared in a proprietary color format. As in analog, converting to an interlaced output like 480i will mean interlacing frames.
The Wii U will prepare data for the HDMI connection. This supports 480i, 480p, 720p, and 1080p. - Common Tasks
- Scaling
If the rendering resolution is lower than the output resolution, the the chip will upscale to the target output resolution. If the output resolution is lower that the rendering resolution, the chip will downscale to the target output resolution.
Downscaling is much more resource intensive than upscaling. If you decide to upscale, it's normally best to upscale using the console rather than your display.
Only the Wii U scales. - Interlacing
Interlacing is the process or taking a full frame and dividing into half. For example, 480i is interlaced. This means 480 lines that are interlaced. This means, the console will first output one frame of just odd lines, then one frame of just even lines. It looks something like this: (Ref 1).
The output chip will take the full frame, interlace it and then convert it to the appropriate analog or digital signal. This of course adds a small amount of latency but very negligible. Interlacing is not needed for any of the progressive scan outputs (480p, 720p and 1080p). Therefore in these formats, the process of interlacing is skipped.
- Scaling
- Analog
- Output Connection
GameCube Analog - 480i
GameCube Digital¹ - 480i, 480p
Wii Analog - 480i, 480p
Wii U Analog - 480i, 480p², 720p², 1080i², 1080p²
Wii U Digital - 480i, 480p, 720p, 1080i, 1080p
¹ The GameCube's digital port outputs in a proprietary Nintendo video format but output is still 480i & 480p.
² Requires Component Cable - Cables
GameCube Analog - Composite, SVideo, Component¹
GameCube Digital - Component², HDMI³
Wii Analog - Composite, SVideo, Component
Wii U Analog - Composite, SVideo, Component
Wii U Digital - HDMI
¹ You can find unofficial component cables for the GameCube but the analog port will only output 480i
² GameCube Digital AV port outputs a special digital format. Because it is proprietary and undocumented, there are no after market cables made for it. The only option is to buy a official Nintendo Component Cable. This cable actually has a Digital to Analog chip on the connector that converts the data to a YPbPr analog connection outputting either 480i or 480p. This is why the cable is so expensive and has a thick connector.
³ There is currently work being done on using the raw data to convert straight to HDMI. There's a project that uses the raw RGB colorspace from the Digital AV Port to output to native HDMI with no analog conversion needed and the simplest upscaling possible. It'll look like this and the prototype looks like this.
With the exception of the Japanese only D-Sub cable which has the same DAC chip, there are no other known uses for GameCube Digital port. The chip inside both this and the Component cables can be modified to output RGBHV instead of YPbPr to use a VGA connector instead. - Displays
- CRT
CRT displays are analog and will shoot (literally) the image pretty much how it's being piped over. CRTs shoot electron guns based on the analog signal being received. The amount of latency present is pretty much measured with how long it takes the CRT to split the signal to fire off the electron guns. This is normally 1 ms or less.
Old CRT TVs only output at 480i. Common inputs are Composite, SVideo. Sometimes Component is present but 480p will not display.
CRT PC monitors can support multiple resolutions and at minimum 480p. CRT monitors may not support 480i. Also few CRT monitors support any other format besides RGBHV (VGA or BNC connectors).
HDTV CRTs commonly support 480i, 480p, 720p and 1080i. Due to their massive size, they are unpopular. Because they are made for media playback, they may still be victim to post processing latency. They commonly have composite, svideo and component input. Some may have HDMI but since CRTs are analog, the signal will be converter from digital to analog before displaying. - LCD
LCDs are digital and have a fixed amount of points called pixels. They are grid shaped and can display 1:1 exactly what's being rendered. They are designed to display digital color information onto a grid.- Analog to Digital Conversion
Any video data that is received though an analog connection must be converted to a digital format. This is called Analog to Digital Conversion (ADC).
Any LCD display that accepts analog connections like Composite, SVideo, Component or VGA will convert data piped through into a digital format to map it to corresponding pixels on the display. Some older LCD displays have the exact same pixels are those being rendered by the GameCube and Wii consoles and have a resolution of 640x480. HDTVs have many more pixels and because of this, the image will to be scaled to fill the screen. - Upscaling
Upscaling is the process of converting a smaller image into a larger image using a usually constant scale. Upscaling is a rather trivial process. It's simply multiplication which processors do pretty easily. For example, converting 480p to 1080p means multiplying the x and y coordinates by 2.25 (1080p/480p). Unfortunately, some scalers might do some fancy scaling algorithms on the image which make things slower. If your LCD set allow you to set the zoom or scaling to 1x and it just renders a little box in the center, this means you've turned off scaling. By playing with this option, you can normally see that scaling isn't very noticeable in terms of latency.
If scaling is built into the ADC (Analog to Digital Converter) chip then things are smoother but, if not, that data has to be piped over to the scaling chip. This is not to mention the latency involved in the ADC conversion process.
Theoretically this is all done in less than a millisecond or two. The only issue is when it's not built for doing things quickly or when there is post processing. - Post-processing
Post-processing is the act of taking an fully scaled image and apply little effects to it like dynamic contrast. This is an extra added latency and not necessary at all and simply cosmetic. - Screen Output
After the signal is converted to digital and scaled accordingly and all the post-processing, if any, completed, now the LCD screen will start sending electrical charges to each pixel (and subpixel). Now, remember, there are 307,200 (640x480) pixels in a low resolution set and 2,073,600 in a 1080p set. Now there's a so many different ways to measure how fast an LCD screen is at displaying the data that there are different terms. *Response Time* is the time it takes for a pixel to go from complete black (0 or completely off) to completely white (255 or completely on) and back to off again. This would be 0-255-0 in terms of brightness. Then there's *Gray to Gray* (GTG) which is the time it takes to go from gray to white to gray. This is something like 128-255-128. Typical Response Time is 8 to 16ms and typical GTG is 2 to 6ms. Obviously GTG is shorter and this is what's used commonly when marketing. It's hard to find Response Time listed with LCD displays now. - Other notes
Response time is just a part of the reason for lag but remember the other parts. There's scaling, ADC, and post processing.
LCD displays designed for the use as computer monitors generally exhibit less lag than those built for media playback. This is mostly because trying to use a mouse on a laggy display is very uncomfortable. Most LCD TVs that allow you define an input as PC will disable post processing for this reason.
- Analog to Digital Conversion
- CRT
To clarify, the term Input Lag means the time it takes for what you do on your input device (your GameCube controller) to appear on the screen. Sometimes it's defined as the time it takes a device to display what's being received from the connectors, but it generally should be everything together. (Wavebirds, converters, deinterlacers, upscalers, etc).
As for input lag, the reality is, complains about input lag has near nothing to do with reaction time. No one is going to react to something an input something a frame later. The brain doesn't really work that way the average response time is something about 100ms while a good response time is 30-50ms (world record is 10ms).
Latency when playing really only matters to timed responses like powershielding/perfectshielding an attack or Smash DI input. With a 1 frame lag on a display one could arguably state that input window is shortened by 1 frames. (In reality, it's offset by 1 frames).
- Example
For example, say you were to powershield a spammy Falco's laser. Powershielding is 6 frame process (with a 2 to 4 frame window depending on projectile or physical attack).
This is how Melee processes things.
Frame #|Effect
01|Nothing
02|Nothing
03|Reflect / PowerShield
04|Reflect / PowerShield
05|PowerShield
06|PowerShield
7+|Regular Shield
Now, on a 2 frame lag display, everything is 2 frames later, so when you think you pressed Shield on frame 4, it actually happened on frame 6. No reflect for you.
What would happen is, if you play long enough on the laggy LCD set, you will start inputting things 2 frames EARLIER than what you would on a "lagless" setup. This honestly isn't a problem if EVERYONE used the same setup. The problem is when you play and train on LCD and then try to play on a CRT, say, like at a tournament. Everything will start not working properly. Now your powershielding window is pretty much cut from 4 frames to 2 and you'll never reflect anything.
- To Consider
The reason why people can play fighting games on LCDs is because there is a standard for it. They all pretty much use the EVO monitor. Everyone should be using EXACTLY the same monitor lag. I'm sure you can also see a problem to coming from this. If, for example, everyone playing SF4 used a monitor with input lag of 10ms (which is about much the Asus monitors at EVO lag), and then for EVO2014, they use a 1ms lag, everyone will have already been used to playing with 9ms laggier set. Of course, 9ms is still shorter than a frame, but there is still some room for error.
With Melee and Brawl, we already have a standard, and that standard is sub-1ms lag from CRTs. Smash4 will be a bit of a mess unless we standardize exactly what latency we're going to play with. Hopefully, OLED displays will be cheaper and more available by the time Smash4 comes out since they will have sub-1ms lag. If not, then, we might just have to play at 480i/480p CRTs or all agree to play on a certain monitor.
Adapters/Converters
The following is a list of converters/adapters with highly negligible lag (nanoseconds)
(Source) - (Destination)
Composite - Composite, SVideo, Component
SVideo - Composite, SVideo, Component
Component - Composite¹, SVideo¹, Component², VGA³
VGA - Component³, VGA
DVI - DVI, HDMI
HDMI - DVI, HDMI
¹ Format must be 480i
² Format cannot be changed
³ Format must be 480p, 720p or 1080p
Any conversion from an analog signal (Component) to a digital signal (HDMI) will incur Analog to Digital Conversion (ADC) lag.
Any conversion from an Interlaced (480i) format to a Progress Scan format (480p) will incur deinterlacing lag.
Any conversion from a higher resolution (1080p) to a lower resolution (480p) will incur downscaling lag.
Any conversion from a lower resolution (1080p) to a higher resolution (480p) will incur upscaling lag. Depending on the upscale process, this may be negligible.
FAQs
- Is displaylag.coma reliable source for lag data?
- Yes and no. displaylag.com uses the Leo Bodnar’s Lag Tester has higher inaccurate latency numbers when testing devices. Plasma screens are COMPLETELY inaccurate and should be considered. Even though the numbers are inaccurate, it can be used as a reference to compare similar LCD displays to see which are faster than others.
- If displaylag.com isn't reliable, what can I do to get an accurate representation of lag?
- You can either use hdtvtest's extremely limited database or conduct your own lag test with a camera.
- How do I conduct my own lag test?
- More on that later...
- This whole thing is entirely too technical for me, what's the best advice for me to play on an LCD.
- If you have using an LCD screen, don't even play with a 480i connection like composite because the deinterlacing process can really lag. You should only be using component cables. If you use a GameCube, this may be hard to find, but for the Wii, component cables are readily available.
- How to enable 480p?
- For Melee, you have to hold B when you start the game. You'll be asked if you want to turn on progressive scan. Select yes.
- For the Brawl/P:M, you have to enable 480p in the Wii console settings.
- Where can I post more questions?
- In this thread OR on reddit
