Sub-pixel interpolation filter design and combined inter-intra prediction for video coding

During the past two decades, there have been many developments in multimedia representation and communications. However, the amount of uncompressed video data is too large for the limited transmission bandwidth or storage capacities. Therefore, video compression becomes an essential component of multimedia services. State-of-the-art video coding standard H.264/AVC applies motion-compensated prediction in combination with transform coding of the prediction error. Motion-compensated prediction is very efficient for removing the correlation between pixels in temporal domain. For inter prediction coding, H.264/AVC has developed several new techniques: variable block-size motion compensation, quarter-sample-accurate motion vector and multiple reference pictures. In order to estimate and compensate the fractional-pel displacements, reference frame is interpolated using a 6-tap Wiener interpolation filter for half-pixels and bilinear filter for quarter-pixels. However, the interpolated sub-pixels are not accurate enough. To further improve the coding efficiency, novel adaptive sub-pixel interpolation filters are proposed in this thesis. Considering the local image characteristics, the first method designs interpolation filters for sub-pixels in low-frequency and high-frequency areas separately. And in order to decrease the header information, flexible symmetry is assumed for each filter. Experimental results indicate that this method achieves up to 0.39dB coding gain which equals to an 11.39% average bit-rate reduction for high-resolution video materials compared to the standard non-adaptive interpolation method of H.264. Compared to the state-of-art 2D non-separable adaptive interpolation filter, an average bit-rate saving of 0.83% is achieved for high-definition video coding. Later, another localized sub-pixel interpolation filter LAIF is proposed in which for certain fractional-pixel locations in the reference frame, filter with adaptive interpolation window is applied. It is shown that this improved method has an average bit-rate saving of 11.06% compared with the H.264/AVC. Besides temporal correlation existed among consecutive frames, spatial correlation between neighboring pixels can also be exploited by intra-prediction coding. Previously, it has been shown that smaller prediction error compared to H.264/AVC can be produced by combining inter and intra prediction together to generate a more accurate prediction signal. In this thesis, we further improve the combined prediction scheme to enable weighted coefficients for inter and intra prediction to be tuned adaptively to local signal characteristics. First, edge detection on the inter prediction block of current macroblock is performed to choose which intra prediction mode to combine with. Then, spatial-variant weighted coefficient is determined for each combined mode. In order to avoid additional overhead signaling, statistics of previously reconstructed macroblocks is analyzed to predict the weighted coefficients of the combined prediction for current macroblock. Compared to H.264/AVC, bit-rate reduction is increased by up to 2.235%. And compared to the latest combined prediction method using spatial-invariant weighted coefficients, simulation results indicate that the proposed scheme achieves additional coding gain of up to 0.766%. Besides, a complexity-reduced scheme is further presented, which allocates fixed but still spatial-variant weighted coefficient for each combined mode, and numerical results indicate that it achieves a better trade-off between coding efficiency and computational complexity