Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model

Capturing general deforming scenes is crucial for many computer graphics andvision applications, and it is especially challenging when only a monocular RGBvideo of the scene is available. Competing methods assume dense point tracks,3D templates, large-scale training datasets, or only capture small-scaledeformations. In contrast to those, our method, Ub4D, makes none of theseassumptions while outperforming the previous state of the art in challengingscenarios. Our technique includes two new, in the context of non-rigid 3Dreconstruction, components, i.e., 1) A coordinate-based and implicit neuralrepresentation for non-rigid scenes, which enables an unbiased reconstructionof dynamic scenes, and 2) A novel dynamic scene flow loss, which enables thereconstruction of larger deformations. Results on our new dataset, which willbe made publicly available, demonstrate the clear improvement over the state ofthe art in terms of surface reconstruction accuracy and robustness to largedeformations. Visit the project page https://4dqv.mpi-inf.mpg.de/Ub4D/.