Visual comfort of 3-D TV : models and measurements

The embracing of 3-D movies by Hollywood and fast LCD panels finally enable the home consumer market to start successful campaigns to get 3-D movies and games in the comfort of the living room. By introducing three-dimensional television (3-D TV) and its desktop-counterpart for gaming and internet applications on the public consumer market, viewers will be provided with a whole new experience. The difference between 3-D TV and its predecessor is the introduction of binocular disparity, i.e., the fact that the left and the right eye receive a slightly horizontally shifted perspective of the same scene, from which the brain extracts depth information. As a consequence, the viewer perceives the image as if its content is positioned in three-dimensional space, i.e., both in front of and behind the television screen. Central to these developments are be the viewer’s experiences which will signify the success or failure of proposed innovative imaging technology, i.e., both perceived image quality and viewing comfort should be at least comparable to conventional television. The aim of this thesis is therefore to understand, measure and eventually, model and predict the added value of stereoscopic depth as well as the accompanying visual discomfort associated with 3-D TV. With respect to the added value of stereoscopic depth, a 3-D Quality Model as an extension of Engeldrum's Image Quality Circle is proposed in Chapter 2, since many of our results confirm that the added value of stereoscopic depth is not captured with the Image Quality Circle. The 3-D Quality Model describes higher level evaluation metrics as a weighted sum of image quality and depth. Two higher level evaluation metrics are investigated; naturalness and viewing experience. In experiments 1, 2 and 3 image quality (levels of noise and blur) and stereoscopic depth (different camera base distances and levels of screen disparities) are varied and evaluated in terms of image quality, depth, viewing experience and naturalness. The results show that perceived image quality and perceived depth are not independent in their relationship to physical image characteristics, but are on the perceptual level. Variations in image quality are reflected by viewing experience and naturalness to a similar extent, yet the added value of stereoscopic depth is more incorporated in naturalness. Naturalness is most appropriate to evaluate the added value of 3-D quality of stereoscopic stills, since it weights stereoscopic depth most in addition to image quality. An important aspect is that these results are consistent over a range of different 3-D displays and content generation methods (thereby reflecting different depth percepts) and image quality attributes. Hence, we have shown that the 3-D Quality Model based on naturalness as evaluation metric is validly applicable to stereoscopic stills and that its value is determined for approximately 75% by image quality and for approximately 25% by the added value of stereoscopic depth. With respect the accompanying visual discomfort associated with 3-D TV a first step is to arrive at a full understanding of visual discomfort, its determinants and contributing factors, and the measurable effects it has on viewers’ visual functioning and subjective experience. In line with this, a theoretical frame work is presented in Chapter 3 on which our research concerning the visual discomfort is centered and that reduces the ambiguity of the visual discomfort associated with stereoscopic display technology and image generation. Our general recommendation is to adhere to a ‘one degree of disparity’ limit. This limit allows for sufficient depth rendering for most application purposes and should guarantee a comfortable viewing in stereoscopic television. This range is based on human vision characteristics; screen disparities can be fused and the buffer in our accommodation-vergence system accounts for mismatches. Within this zone of comfortable viewing, visual discomfort can still occur to an extent, however, which is likely to be caused by one or more of the following three factors: excessive demand of accommodation-convergence linkage; 3-D artifacts; and unnatural amounts of blur. Chapter 3 also describes potential measurement methods to evaluate the visual discomfort associated with stereoscopic viewing. We define visual fatigue as physiological strain or stress resulting from excessive exertion of the visual system, which can be objectively measured and visual discomfort as its subjective counterpart. To adequately characterize and determine the degree of visual fatigue and visual discomfort in a sensitive, accurate, reliable and valid way, multiple indicators for both components can be relevant. Visual discomfort can be evaluated with validated questionnaires or other self-report measures. For visual fatigue clinical measurement methods are best suited since they are relatively cheap, concise, quantitative with a high sensitivity and specificity and applicable to a large group of participants. In Chapter 4 knowledge is gained on how visual discomfort is built up whilst watching stereoscopic content within the ‘one degree of disparity’ limit. The aim was to determine which video characteristics, e.g., lateral object and camera motion and (changes in) disparity, induce visual discomfort. The values of such video characteristics can be extracted from stereoscopic movies with motion and depth estimation algorithms and directly related to a continuous assessment (CA) in terms of visual discomfort. Hence, a CA of a long-term stereoscopic movie in terms of visual comfort can provide valuable moment-to-moment information concerning the perceptual impact of specific video characteristics. Two experiments (experiments 4 and 5) were designed with the primary objective of relating the impact of these video characteristics to the assessment of visual discomfort. In addition, experiment 4 compares a continuous assessment method with other assessment methods of visual comfort, and experiment 5 investigates the impact of subtitles on visual comfort. Three 3-D movies were assessed in terms of visual comfort via a continuous assessment. The continuous assessment scores were directly compared to video characteristics that were derived from the 3-D movies. Additional assessment methods in experiment 4 included the assessment of six 10-second sequences captured from the 3-D movie and a single retrospective assessment of the entire 3-D movie. The two 3-D movies in experiment 5 were shown with or without subtitles. Results show that the visual comfort of stereoscopic scenes can be predicted as a linear combination of screen disparity range and offset, changing screen disparity, and lateral motion. The specific contributions of these characteristics depend on the scene, yet more complex models are required to extend the comfort prediction to entire movies, incorporating different scenes. In addition, the results of experiment 5 reveal that subtitles required additional effort to keep vision comfortable and the results of experiment 4 show that the correlation between the assessment of the 10-second sequences captured from the 3-D movie and their corresponding parts within the continuous assessment is low, whereas the correlation between the retrospective assessment and the mean of the continuous assessment score over scene parts with a high screen disparity is higher. The aim of Chapter 5 is to construct a measurement protocol for the evaluation of objective signs of visual fatigue and subjective symptoms of visual comfort. It is known that some people have a sensitive binocular vision and their visual system is challenged too much under unnatural viewing conditions. Consequently, they may experience visual discomfort regardless the absence of visual discomfort factors in the 3-D content. To adequately characterize the degree of objective visual fatigue and subjective visual discomfort, a measurement protocol was constructed that also incorporated differences in performance of the binocular system. In experiment 6 the categorization of participants’ binocular status is accomplished based on an optometric algorithm that reflects the values of ten single optometric tests; one subgroup with a good binocular status and one with a moderate binocular status. The main objective of experiment 6 is to identify the most appropriate optometric indicators for visual discomfort and visual fatigue. Objective and subjective optometric tests were applied before and after short-term stressful stereoscopic reading tasks both in 2-D and 3-D. The reading tasks were different passages of the Wilkins Rate of Reading test (an optometric test). Results reveal that participants with a moderate binocular status are more susceptible to visual discomfort associated with stereoscopic content based on objective signs of visual fatigue in the fusion range and the subjective symptoms of visual discomfort. In addition, the results reveal the number of words read in 3-D relative to number of words read in 2-D, referred to as WRRT-ratio, is indicative of the binocular status of people. In experiment 7 the aim is to confirm that the WRRT-ratio is a proper indicator of the binocular status of people and thus of their susceptibility to visual discomfort. Similar as in experiment 1, the categorization of participants’ binocular status is accomplished based on the optometric algorithm. The results reveal that even though the specific WRRT-ratio depends on viewing conditions, it is consistent with susceptibility: participants that have a moderate binocular status have poorer reading performance in 3-D than in 2-D and experienced more visual discomfort compared to people with normal binocular vision. In experiment 8 the objective is to determine whether the threshold in screen disparity for visual discomfort is related to the binocular status of people. The categorization of participants’ binocular status is accomplished based on the WRRT-ratio and objective and subjective evaluation is performed by the fusion range and questionnaires respectively. The results show that only participants with a moderate binocular status reveal a tendency in changed fusion range. These participants also indicate significantly more visual discomfort in stereoscopic conditions and set lower thresholds in screen disparity for visual discomfort than participants with a good binocular status. Hence, a combination of fusion range measurements and self-report is appropriate for evaluating visual complaints. The results also show that the WRRT-ratio is indicative of the binocular vision of people. This WRRT-ratio is very promising, since 1) it can be used to explain inter-subject differences in results, 2) it provides additional information to stereo- and visual acuity tests, and 3) it can be implemented in 3D commercial displays to advise people with moderate binocular vision to lower the screen disparity range in order to avoid visual complaints. Chapter 5 briefly look back at the previous chapters by summarizing and discussing the most important findings and presents the applicability of the main findings as well as a future view and research that is left for future research to perform