Similarity-Aware Patchwork Assembly for Depth Image Super-Resolution

This paper describes a patchwork assembly algorithm for depth image super-resolution. An input low resolution depth image is disassembled into parts by matching similar regions on a set of high resolution training images, and a super-resolution image is then assembled using these cor-responding matched counterparts. We convert the super-resolution problem into a Markov Random Field (MRF) la-beling problem, and propose a unified formulation embed-ding (1) the consistency between the resolution enhanced image and the original input, (2) the similarity of disas-sembled parts with the corresponding regions on training images, (3) the depth smoothness in local neighborhoods, (4) the additional geometric constraints from self-similar structures in the scene, and (5) the boundary coincidence between the resolution enhanced depth image and an op-tional aligned high resolution intensity image. Experimen-tal results on both synthetic and real-world data demon-strate that the proposed algorithm is capable of recovering high quality depth images with ×4 resolution enhancement along each coordinate direction, and that it outperforms state-of-the-arts [14] in both qualitative and quantitative evaluations. 1