Probabilistic Deformation Models for Biometric Image Matching in Harsh Environments

This thesis demonstrates techniques for improved biometric image matching by mitigating image distortions in real world scenarios, including limited and noisy data. In particular, decreases in data quality frequently stemming from uncontrolled acquisition environments (e.g., unconstrained users), and/or distortions in the biometric patterns (due to factors such as occlusion, pose variation, or facial expression changes can significantly degrade system performance. We argue that in order to improve recognition performance in these challenging scenarios we must effectively leverage how the discrimination ability varies across the pair of images being matched. To this end we introduce several novel developments and design a newmatching framework that considers the discrimination ability of each image region at each step of matching.We argue that estimating the non-stationary deformations between the unknown image sample (referred to as the ‘probe’) and stored template (referred to as the‘gallery’) at a region-level is more robust against high-distortion matching conditions than accounting for deformations at a fine level (i.e., quantifying individualpixel shifts required to best fit one image to another). Accordingly, we introduce an unsupervised approach to automatically select said regions to use to divide the probeand gallery images based solely on discrimination ability.Using cross-correlation, the proposed model is then able to simultaneously estimate both the deformation cues (i.e., the x y translation of each region) and similarity cues (i.e., the match score for each relative shift between the corresponding regions). However, within difficult matching scenarios an authentic correlation output may be difficult to discern from an impostor output. We address this by introducing a novel approach for the implementation and design of correlation filters (CFs) as classifiers. Referred to as ’Stacked Correlation Filters’ (SCFs), this architectureconsists of training a series of stacked modular CFs with each layer refining the output of the previous layer. As previous works with CFs have only focusedon individual filter design or application, SCFs represent a new paradigm in the CF community.Finally, we employ a Bayesian graphical model to estimate the non-stationary deformations between a given probe image and gallery template and find the maximumscoring assignment for the match pair. We argue that with the derived mapping between an input comparison and output deformation score, a Gaussian ConditionalRandom Field (GCRF) can effectively capture the deformations found within the training set using a parameter estimation scheme that does not employ inference.We thoroughly analyze the performance of the proposed model via extensive experimentation on challenging data. We conduct experiments on large-scale biometricdatasets, comprised of over 62000 images from over 1000 different subjects, emulating ’in-the-wild’ matching over varying sensors, acquisition environments, andbiometric modalities. In total, we complete more than 200 million image comparisons in challenging scenarios leading to state-of-the-art verification performance