Depth map inpainting and super-resolution with arbitrary scale factors

Depth information is useful for many applications in the real world such as view synthesis, and 3D reconstruction. However, the resolution of the depth map is much lower than the resolution of the color image. Therefore, super-resolution on depth maps has drawn a lot of attention and achieved great success with machine learning-based methods in recent years. However, the prior work is still limited to training a specific model for each integer scale factor and only works with a set of few integer scale factors. Moreover, the depth map collected by the depth sensor can also have a lot of depth missing values and depth error along the edge and corners of observed objects. In this work, we propose a novel method to perform both super-resolution by an arbitrary scale factor and inpainting to fill in the missing depth values on the depth map. The experimental results on Middlebury RGBD datasets show the effectiveness of our proposed approach