An overview of research in 3DTV

3DTV is regarded by the experts and the general public as the next major step in video technologies. The ghost-like images of remote persons or objects are already depicted in many futuristic movies; both entertainment applications, as well as 3D video telephony, are among the commonly imagined utilizations of such a technology. As in every product, there are various different technological approaches also in 3DTV. By the way, 3D technologies are not new; the earliest 3DTV application is demon-strated within a few years after the invention of 2D TV. However, earlier 3D video relied on stereoscopy. Current work mostly focuses on advanced variants of stereoscopic principles like goggle-free autostereoscopic multi-view devices. However, holographic 3DTV and its variants are the ultimate goal and will yield the envisioned high-quality ghost-like replicas of original scenes once technological problems are solved. Stereoscopy is based on exploiting the human perception. Simply, two views, taken at two slightly different angles are then guided to left and right eyes. The two eyes, receiving the two different views of the same scene from two different angles, provide the visual signals to the brain; and then, the brain interprets the scene as 3D. However, there are many different 3D depth cues in perception, and usually, there are contradictory signals received by the brain. Viewers experience a motion-sickness-like feeling as a consequence of such mismatches. This is the major reason which kept 3D from becoming a popular mode of visual communications. However, recent advances in end-to-end digital techniques minimized such problems. Stereoscopic TV broadcasts have been conducted. Novel advances in stereoscopy brought viewing without goggles; however, the viewer and the monitor must have a fixed location and orientation with respect to each other for most autostereoscopic images. Multi-view autostereoscopic displays allow some horizontal parallax within a limited viewing angle. There are experiments in head-tracking autostereoscopic displays, as well as, free-view point video by providing the right pair of images based on the location of the viewer. Holography is not based on human perception, but targets perfect recording and reconstruction of light with all its properties. If such a reconstruction is achieved, the viewer, embedded in the same light distributionas the original, will of course see the same scene as the original. Classical holography tries to achieve this by recording the amplitude and phase distribution of light over a surface; this surface, when illumiated, yields the same physical light in space. Similar to holography, integral imaging targets to record, and play back the amplitude and direction information of light passing through a surface. Since both techniques target the physical duplication of light, integral imaging and classical holography may be classified under general holography. Holographic 3DTV can be achieved if the holographic recordings and the associated holographic display can be refreshed in real-time. Currently, dynamic holographic capture by CCD arrays, and dynamic holographic display by spatial light modulators (SLMs) are demonstrated. However, due to limited number of array elements, and limitations regarding the pixel sizes, such holographic 3DTV displays have a very small angle of view (about 2 degrees), and therefore, far from being satisfactory at present. Applications of 3D video technologies to different fields like medicine, dentistry, navigation, cultural exhibits, art, science, education, etc., in addition to primary application of entertainment and communications, will revolutionarize the way we interact with visual data, and will bring many benefits. A consortium of 19 European institutions, led by Bilkent University, has been focusing on all technical aspects of 3DTV since Aug 2004: 3D scene capture, representation, compression, transmission and display are the main technical building blocks. Fundamental signal processing issues associated with scalar wave propagation, diffraction and holography are also of prime interest. It is envisioned that future 3DTV systems will decouple the capture and display steps: 3D scenes will be captured by some means, like multi-camera systems, and this data will then be converted to abstract 3D representation using computer graphics techniques. The display will then access this abstract data to generate the 3D video to the observer