MAP based motion field refinement methods for motion-compensated frame interpolation

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2014. 2. 김태정.In this dissertation, maximum a posteriori probability (MAP) based motion refinement methods are proposed for block-based motion-compensated frame interpolation (MCFI). The first method, called single hypothesis Bayesian approach (SHBA), is aiming at estimating the true MVF of a video frame from its observed MVF, which is the result of a block-based motion estimation (BME), by maximizing the posterior probability of the true MVF. For the estimation, the observed MVF is assumed to be a degraded version of the true MVF by locally stationary additive Gaussian noise (AGN), so the variance of the noise represents the unreliability of the observed MV. The noise variance is directly estimated from the observation vector and its select neighbors. The prior distribution of the true MVF is designed to rely on the distances between the MV and its neighbors and to properly smooth false MVs in the observation. The second algorithm, called multiple hypotheses Bayesian approach (MHBA), estimates the true MVF of a video frame from its multiple observations by maximizing the posterior probability of the true. The multiple observations, which are the results of a BME incorporating blocks of different sizes for matching, are assumed to be degraded versions of the true MVF by locally stationary AGN. The noise variances for the observations are first estimated independently and then adaptively adjusted by block-matching errors in order to solve motion boundary problem. Finally, a method, called single hypothesis Bayesian approach in a bidirectional framework (SHBA-BF), that simultaneously estimates the true forward and backward MVFs of two consecutive frames from the observed forward and backward MVFs is proposed. The observed MVFs are assumed to be degraded versions of the corresponding true MVFs by locally stationary AGN. The true forward and backward MVFs are assumed to follow the proposed joint prior distribution, which is designed such that it adaptively relies on not only the resemblance between spatially neighboring MVs but also the resemblance between the MV and its dual MV so the proposed simultaneous estimation can fully exploit duality of MVF. Experimental results show that the proposed algorithms obtain better or comparable performances as compared to the state-of-the-art BME algorithms at much lower computational complexity.Abstract i Contents iii List of Figures v List of Tables x 1 Introudction 1 1.1 Motion-Compensated Frame Interpolation . . . . . . . . . . . . . . . 1 1.2 Previous Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2.1 Exploit spatio-temporal correlation during ME . . . . . . . . 3 1.2.2 Utilize multiple block sizes for matching in ME . . . . . . . . 6 1.2.3 Correct false motion vectors in given MVFs . . . . . . . . . . 7 1.3 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Single Hypothesis Bayesian Approach 11 2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Proposed observation likelihood . . . . . . . . . . . . . . . . 12 2.1.2 New prior distribution for true motion vector field . . . . . . . 15 2.2 Estimation of AGN Variance . . . . . . . . . . . . . . . . . . . . . . 23 2.2.1 Proposed covariance matrix estimation method . . . . . . . . 24 iii 2.2.2 Performance of the proposed reliability measure . . . . . . . 31 2.3 Solution to MAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.4 Relation to Previous Works . . . . . . . . . . . . . . . . . . . . . . . 34 2.5 Properties of Proposed Prior Distribution . . . . . . . . . . . . . . . . 36 3 Multiple Hypotheses Bayesian Approach 38 3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.1.1 Proposed observation likelihood . . . . . . . . . . . . . . . . 39 3.1.2 Prior distribution of true motion vector field . . . . . . . . . . 43 3.2 MAP Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3 Adaptive Adjustment of Estimated Noise Variances . . . . . . . . . . 45 4 Single Hypothesis Bayesian Approach in a Bidirectional Framework 50 4.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.1.1 Observation likelihood . . . . . . . . . . . . . . . . . . . . . 51 4.1.2 Joint prior distribution of true motion vector fields . . . . . . 52 4.2 MAP Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5 Experimental Results 56 5.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.2 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.2.1 Performance of SHBA . . . . . . . . . . . . . . . . . . . . . 60 5.2.2 Performance of MHBA . . . . . . . . . . . . . . . . . . . . . 71 5.2.3 Performance of SHBA-BF . . . . . . . . . . . . . . . . . . . 72 6 Conclusion 89 Abstract In Korean 98Docto