Deinterlacing is the process of converting interlaced video, like common analog television signals, into a non-interlaced form.
Video (or film) consists of a series of images played in succession, each of these images is known as a frame. For a variety of reasons, analog television standards break these frames into a number of lower-resolution fields, each one containing a portion of the image. The fields are then displayed in a fashion that produces the illusion of single frames again. This process of producing complete frames from the lower-resolution fields is known as interlacing. For instance, in the NTSC system used in North America and Japan, frames are made for every 30th of a second, but are recorded as a series of 60th second fields, each containing only the odd or even lines of the display.
Interlacing does not work well with modern display technologies, such as LCDs. Deinterlacing attempts to reverse the interlacing process to produce single higher resolution frames suitable for these displays. However, the original interlaced signal did not contain these frames in a complete form, and de-interlacing is a fundamentally lossy process that generally introduces some image degradation. The exception to this rule is deinterlacing an originally non-interlaced source that was later interlaced for display; movies encoded using telecine are one example.
Basic concept
In a typical CRT television, the interlaced display draws only half of the lines on the screen for each frame, alternately drawing the odd and even lines for each frame. This reduces flicker by taking advantage of the persistence of vision effect, producing a refresh rate of double the frame rate without the overhead of either transmitting each frame twice or holding it in a buffer so it can be redrawn.
When displaying video on a display that can support a high enough refresh rate such that flicker isn't perceivable, interlaced video can be deinterlaced for better viewing; likewise when a display cannot interlace but must draw the entire screen each time, the video must be deinterlaced before it can be displayed. All current displays except for CRT screens require deinterlacing. Though in principle there is no reason why LCD, DLP or Plasma displays should not display two fields sequentially, the requirement that half of the pixels remain black would result in a halving of brightness, even assuming that the response time of the technology could be made fast enough.
Deinterlacing requires the display to buffer one or more fields and recombine them into a single frame. In theory this would be as simple as capturing one field and combining it with the next field to be received, producing a single frame. However the originally recorded signal was produced as a series of fields, and any motion of the subjects during the short period between the fields is encoded into the display. When combined into a single frame, the slight differences between the two fields due to this motion results in a "tearing" effect where alternate lines are slightly displaced from each other.
Modern deinterlacing systems therefore buffer several fields and use techniques like edge detection in an attempt to find the motion between the fields. This is then used to interpolate the missing lines from the original field, reducing the "tearing" effects.
Deinterlacing methods
There are various methods to deinterlace video, each producing different artifacts. Artifacts will always be present in deinterlaced video, as the process must attempt to combine two fields for simultaneous presentation. Any object that is moving will appear in different positions on the two fields, and simply displaying them overlaid results in very objectionable mouse teeth, venetian blinds, or 'comb-effect' on the moving vertical edges. There is no perfect way to interpolate images in time, unless everything is moving together, as in a panned image.
There are two basic methods of deinterlacing: combination, where the even and odd frames are combined into one image and then displayed, and extension, where each frame (with only half the lines) is extended to the entire screen.
Field Combination Deinterlacing
- Weaving is done by adding consecutive fields together. This is fine when the image hasn't changed between fields, but any change will result in artifacts known as "mouse teeth" or "tearing", when the pixels in one frame do not line up with the pixels in the other, forming a jagged edge. This technique retains full vertical resolution at the expense of half the temporal resolution.
- Blending is done by blending, or averaging consecutive fields to be displayed as one frame. The mouse teeth are avoided because both of the images are on top of each other. This instead leaves an artifact known as ghosting. The image loses vertical resolution and temporal resolution. This is often combined with a vertical resize so that the output has no numerical loss in vertical resolution. The problem with this is that there is a quality loss, because the image has been downsized then upsized. This loss in detail makes the image look softer.
- Selective blending, or smart blending or motion adaptive blending, is a combination of weaving and blending. As areas that haven't changed from frame to frame don't need any processing, the frames are weaved and only the areas that need it are blended. This retains full vertical resolution, half the temporal resolution, and has fewer artifacts than weaving or blending because of the combination of them both.
- Motion Compensation deinterlacing uses motion compensation techniques in order to align the two fields in a picture. While effective for horizontal motion, some software for this technique does not handle vertical motion at all, and may fall back on selective blending or other techniques when it is unable to resolve the motion vectors. This technique is implemented in the open source tool yuvdeinterlace, part of the mjpeg tools package.
- Inverse Telecine: If telecine was used to convert a motion picture to interlaced TV video, then it is possible to reverse the algorithm to obtain the original non-interlaced footage (which has a slower frame rate). In order for this to work, the exact telecine pattern must be known or guessed. Unlike most other deinterlacing methods, when it works inverse telecine perfectly recovers the original progressive video stream.
- Telecide-style algorithms: If the interlaced footage was generated from progressive frames at a slower frame rate (e.g. "cartoon pulldown"), then the exact original frames can be recovered by copying the missing field from a matching previous/next frame. In cases where there is no match (e.g. brief cartoon sequences with an elevated frame rate), then the filter falls back on another deinterlacing method such as blending or line-doubling. This means that the worst case for Telecide is occasional frames with ghosting or reduced resolution. By contrast, when more sophisticated motion-detection algorithms fail, they can introduce pixel artifacts that are unfaithful to the original material. For telecine video, decimation can be applied as a post-process to reduce the frame rate, and this combination is generally more robust than a simple inverse telecine (which fails when differently interlaced footage is spliced together).
Frame Extension Deinterlacing
- Half-sizing displays each interlaced frame on its own, resulting in a video with half the vertical resolution of the original, unscaled. Understandably, this is not used for regular viewing.
- Line doubling takes the lines of each interlaced frame (consisting of only even or odd lines) and doubles them, filling the entire frame. This results in the video having effectively half the vertical resolution, scaled to the full resolution. While this prevents mouse teeth, it causes a noticeable reduction in picture quality. This technique is also called bob deinterlacing, because the fields are bobbed up and down.
Both field combination and frame extension lend themselves to a method called motion compensation. Deinterlacers that use this technique are often superior because they can use information from many fields, as opposed to just one or two. For example, if two fields had a person's head moving to the left, then if weaving was applied, mouse teeth would appear. If blending was applied, ghosting would appear. Selective blending would also create ghosting. Both of the frame extension methods would have no artifacts, but the level of detail on the face would be half. Motion compensation (ideally) would see that the face in both fields is the same, just transposed, and would combine the face (through weaving or some other more advanced method) to get full detail in both output frames. This needs to be combined with a scene change detection algorithm, otherwise it will attempt to find motion between two completely different scenes.
The best deinterlacers combine all of the methods mentioned above. The fields are bobbed, so the frame rate is then kept. Motion compensation is done. In the areas that it cannot find a motion match, it falls back on selective blending. The best deinterlacers also determine whether video material contain a 3:2 pulldown sequence in the case of film material, and automatically do a reverse telecine instead of the above deinterlacing techniques. This operation is made automatic in modern deinterlacers, depending on whether the video is currently film-source or not, by automatically detecting the presence or absence of a 3:2 pulldown sequence.
Interlacing in HDTV
Interlaced video requires the necessarily imperfect process of deinterlacing, and many experts hold that that casts doubt on the wisdom of the choice of 1080i (interlaced) as a standard in television and video (along with 720p (progressive)). The EBU European Broadcasting Union has argued against the use of interlaced video in production, recommending the use of 1080p/50fps (frames per second) as a future production standard for easier conversion to other formats.[1]
While the 1080i/25fps (interlaced) standard has been adopted for broadcasting (along with 720p/50fps progressive) in Europe and some other countries, movies transmitted in the format are necessarily scanned to produce two fields from the same frame, since film contains only 24 frames per second. The double scan might seem to eliminate any necessity for deinterlacing; but this approach may be hard to detect, and requires a determination on the receiving end of which fields should be paired with each other to reproduce the original progressive frames. Detection and proper reassembly can sometimes be difficult when appropriate indicators are not carried through the system. Televisions capable of displaying 1080i/25fps can use field-splitting to display 1080p/25fps in some fashion; although such video suffers the same motion jitter as is apparent in the cinema, it has long been regarded as acceptable for movies. When film content is scanned to split it into fields for pseudo-interlaced transmission, it is also common to blur the material vertically to avoid artifacts on interlaced displays.citation needed Once such a vertical blur has been introduced, it cannot be undone later, so when it is applied it can cause image degradation if the field-splitting process is inverted for use on a progressive-scan display.
Some 1080i HDV camcorders offer progressive mode with cinema-like frame rate of 24 or 25 frames per second, using technique similar to traditional telecine. Progressive frames are packaged within interlaced signal and can be reproduced by standard consumer television equipment.
Deinterlacing is often called line doubling in many cases. There is confusion when this is referred to as interpolation, which uses spatial filtering to generate extra lines and hence reduce the visibility of pixelation on any type of display.[2] Line doubling often refers to the conversion from interlaced to progressive scan, while maintaining vertical resolution. The 'doubling' terminology comes from the fact that an interlaced field contains half the resolution of a full frame. The terminology 'line doubler' is used more frequently in high end consumer electronics, while 'deinterlacing' is used more frequently in the computer and digital video arena. However, both are essentially the same thing in most cases. Line doubling often uses exactly the same algorithms outlined below. For example, when converting 480i to 480p, this can be correctly called line doubling as well. Also, when converting from one resolution to another, multiple steps can ensue. For example, when converting 480i to 720p as in HDTV, a deinterlacing step is typically done first (480i to 480p), before an interpolation step to scale the image.
Where deinterlacing is done
Deinterlacing can be done (if it needs to be) at various points in the chain from filming to watching. When it is done affects the quality of the deinterlace, because the quality of the deinterlacer can vary.
- If it is done in the filming studios, it should be done very well. The people doing it are professionals, and have minimal time constraints. They should also have access to expensive and powerful deinterlacers.
- If it is done at the time of broadcasting, the quality of the deinterlace can vary. It should be organised by professionals, who have a reasonable budget and powerful processors. On the other hand, it needs to be done in real time, so the effort that the deinterlacer can put in is limited by time.
- If it is copied onto a computer and processed there, the quality can also vary immensely, yet (theoretically) a high quality level should be possible, because there are no restrictions on time and there are some very good, free, deinterlacers. However, many people who do this do not know much about deinterlacing, and when making a choice about which deinterlacer and settings to use, will make a random decision. This often causes poor results for video based content as many people are unaware that video has double the temporal resolution of film.
- If done by an embedded electronic device, the quality varies depending on the overall quality of the device. High-quality electronic devices also have high-quality deinterlacers.
See also
References
External links
|
