Super-resolution study via interpolation techniques

xxi, 242 p. : ill. (some col.) ; 30 cm.PolyU Library Call No.: [THS] LG51 .H577P EIE 2014 HungImage interpolation and super-resolution are fundamental research topics of image processing techniques. The objective is to improve the resolution of a single image or a video sequence by exploiting the spatial and temporal correlations of natural images. Image interpolation inserts new pixels between known pixels to increase the resolution, while super-resolution further addresses the blurring effects and noises. In other words, image interpolation can be regarded as a sub-process of super-resolution. The applications of image interpolation and super-resolution are very wide, including medical imaging, image/video coding, image/video zooming, image manipulation, face recognition, view synthesis for 3D video processing, remote sensing, surveillance, etc. Conventional image interpolation algorithms, such as new-edge-directed interpolation (NEDI), soft-decision adaptive interpolation (SAI), use the ordinary least squares (OLS) to interpolate the missing pixels. However, the OLS is not robust to outliers, such that we propose to use the weighted least squares (WLS) to improve the robustness and accuracy of the SAI by weighting the data samples adaptively. Specifically, we empirically analyze the error statistics of data samples to propose a new weighting model for interpolation. Although the image quality improves significantly using WLS, the complexity increases by around three times. To reduce the computational complexity in practical applications, we further replace the WLS by the bilateral filter and apply the fixed-point strategy to estimate the missing pixels, at the expense of a drop of around 0.1 dB in PSNR, but with around 60 times speed up. It requires around just 0.06 second for processing an image of 384{604}256, which is suitable for real-time applications. Image interpolation algorithms are often deployed into different hardware tools that have different computational power. It is very desirable that the interpolation algorithm is computationally scalable to suit for different hardware structures. Conventional computationally scalable algorithms are non-adaptive to the image contents due to their simple design, such that the edges are blurred and jagged. In this thesis, we propose a new noise model to formulate an image interpolation algorithm which is adaptive to the image contents by scaling the number of noise samples to compute the noise covariance matrix. As a result, the proposed scalable algorithm reconstructs much better edges and texture area. Besides the low-resolution degradation, images may suffer from blur and noise. Conventional super-resolution algorithms use the unified Wiener filter for simultaneous interpolation, de-blurring and de-noising. However, the correlation functions used in the conventional algorithms are geometric-based and often ignore the intensity information, such that the edges and textures are not very well reconstructed. We propose to use the nonlocal means, which exploits the nonlocal redundancy according to the image contents, as the correlation function in an iterative way to significantly improve the quality.Note that image or video contents are often coded in the transform domain, i.e., DCT domain. It is more convenient to improve the resolution in the transform domain; however, the conventional algorithms simplify the assumption that the high-frequency transform coefficients are zero, such that the edges and textures are blurred. In this thesis, we propose a new training-based estimator with adaptive linear minimal mean squares to estimate the high-frequency coefficients from the external information in the training sets, such that the proposed estimator improves the PSNR values by 1 dB on average. Nowadays, capturing images and videos are very easy by handheld cameras. It is always desirable to generate a new image from an existing image, which appears as a new snapshot from a different viewing angle and position. This process is called view synthesis; however, the challenge of view synthesis is to fill the occluded area which is not visible in the original image. The current solution is to make use of the depth images from different frames to calculate the weights of linear filters. In this thesis, we propose to use the color information in additional to the depth information to compute the filter weights, in order to significantly improve the visual quality of the filled area. Experimental results show that the proposed algorithms in this thesis perform better than existing approaches in terms of PSNR, SSIM and subjective quality, which justify the motivation of proposed methods. Hence, the research community is advanced in the areas of study of this thesis in a way that the weaknesses of existing approaches are significantly removed. As multimedia applications requiring high quality images or videos are moving much faster than any time before, there are lots of possible directions to further improve the quality of the proposed algorithms. This is a fruitful direction for further research work.Department of Electronic and Information EngineeringPh.D., Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University, 201