Human recognition based on face profiles in video

Face profile is an important aspect of face recognition and it provides a complementary structure of the face that is seen in the non-frontal view. In the past, several methods have been proposed to recognize face profiles in still im-ages. However, face profile images that are captured at a distance by surveillance cameras usually are video se-quences that have a low resolution. It is difficult to extract accurate face profile directly from a low-resolution video frame, which does not have many pixels on the face profile. The emphasis of this paper is to introduce a practical ap-proach for human recognition by using high-resolution face profile images constructed from the low-resolution videos. We use both the spatial and temporal information present in a number of adjacent low-resolution frames of a video sequence to construct high-resolution face profile images. As the quality of high-resolution images relies on the cor-rectness of image alignment between consecutive frames, an elastic registration algorithm is used for face profile im-age alignment. A match statistic is designed to detect and discard poorly aligned images which may degrade the qual-ity of the high-resolution face profile image. After obtaining high-resolution face profile images, we use a dynamic time warping method for face profile recognition. A number of dynamic video sequences are tested to demonstrate the ap-plicability and reliability of our method.