Super-resolution of facial images in video with expression changes

Super-resolution (SR) of facial images from video suf-fers from facial expression changes. Most of the existing SR algorithms for facial images make an unrealistic assump-tion that the “perfect ” registration has been done prior to the SR process. However, the registration is a challenging task for SR with expression changes. This paper proposes a new method for enhancing the resolution of low-resolution (LR) facial image by handling the facial image in a non-rigid manner. It consists of global tracking, local align-ment for precise registration and SR algorithms. A B-spline based Resolution Aware Incremental Free Form Deforma-tion (RAIFFD) model is used to recover a dense local non-rigid flow field. In this scheme, low-resolution image model is explicitly embedded in the optimization function formu-lation to simulate the formation of low resolution image. The results achieved by the proposed approach are signif-icantly better as compared to the SR approaches applied on the whole face image without considering local defor-mations. The results are also compared with two state-of-the-art SR algorithms to show the effectiveness of the ap-proach in super-resolving facial images with local expres-sion changes. 1