Fast Non-Rigid Radiance Fields from Monocularized Data

3D reconstruction and novel view synthesis of dynamic scenes from collectionsof single views recently gained increased attention. Existing work showsimpressive results for synthetic setups and forward-facing real-world data, butis severely limited in the training speed and angular range for generatingnovel views. This paper addresses these limitations and proposes a new methodfor full 360{\deg} novel view synthesis of non-rigidly deforming scenes. At thecore of our method are: 1) An efficient deformation module that decouples theprocessing of spatial and temporal information for acceleration at training andinference time; and 2) A static module representing the canonical scene as afast hash-encoded neural radiance field. We evaluate the proposed approach onthe established synthetic D-NeRF benchmark, that enables efficientreconstruction from a single monocular view per time-frame randomly sampledfrom a full hemisphere. We refer to this form of inputs as monocularized data.To prove its practicality for real-world scenarios, we recorded twelvechallenging sequences with human actors by sampling single frames from asynchronized multi-view rig. In both cases, our method is trained significantlyfaster than previous methods (minutes instead of days) while achieving highervisual accuracy for generated novel views. Our source code and data isavailable at our project pagehttps://graphics.tu-bs.de/publications/kappel2022fast.