Modelling Perceptual Quality and Visual Saliency for Image and Video Communications

The evolution of advanced radio transmission technologies for third and future generation mobile radio systems has paved the way for the delivery of mobile multimedia services. This is further enabled through contemporary video coding standards, such as H.264/AVC, allowing wireless image and video applications to become a reality on modern mobile devices. The extensive amount of data needed to represent the visual content and the scarce channel bandwidth constitute great challenges for network operators to deliver an intended quality of service. Appropriate metrics are thus instrumental for service providers to monitor the quality as experienced by the end user. This thesis focuses on subjective and objective assessment methods of perceived visual quality in image and video communication. The content of the thesis can be broadly divided into four parts. Firstly, the focus is on the development of image quality metrics that predict perceived quality degradations due to transmission errors. The metrics follow the reduced-reference approach, thus, allowing to measure quality loss during image communication with only little overhead as side information. The metrics are designed and validated using subjective quality ratings from two experiments. The distortion assessment performance is further demonstrated through an application for filter design. The second part of the thesis then investigates various methodologies to further improve the quality prediction performance of the metrics. In this respect, several properties of the human visual system are investigated and incorporated into the metric design. It is shown that the quality prediction performance can be considerably improved using these methodologies. The third part is devoted to analysing the impact of the complex distortion patterns on the overall perceived quality, following two goals. Firstly, the confidence of human observers is analysed to identify the difficulties during assessment of the distorted images, showing, that indeed the level of confidence is highly dependent on the level of visual quality. Secondly, the impact of content saliency on the perceived quality is identified using region-of-interest selections and eye tracking data from two independent subjective experiments. It is revealed, that the saliency of the distortion region indeed has an impact on the overall quality perception and also on the viewing behaviour of human observers when rating image quality. Finally, the quality perception of H.264/AVC coded video containing packet loss is analysed based on the results of a combined subjective video quality and eye tracking experiment. It is shown that the distortion location in relation to the content saliency has a tremendous impact on the overall perceived quality. Based on these findings, a framework for saliency aware video quality assessment is proposed that strongly improves the quality prediction performance of existing video quality metrics