Temporal frequency probing for 5D transient analysis of global light transport

(a) We use a photonic mixer device (PMD) coupled with a 2D projector to acquire “light-in-flight ” images and depth maps for scenes with complex indirect transport. (b)-(c) The sharp, evolving wavefronts of light travelling along direct and caustic light paths can be seen clearly. (d) We also recover time-of-flight depth maps in a way that is not affected by strong indirect light. This indirect light mainly consists of diffuse inter-reflections for the scene in (a) but we also show experimental results where caustic light paths dominate. We analyze light propagation in an unknown scene using projectors and cameras that operate at transient timescales. In this new pho-tography regime, the projector emits a spatio-temporal 3D signal and the camera receives a transformed version of it, determined by the set of all light transport paths through the scene and the time de-lays they induce. The underlying 3D-to-3D transformation encodes scene geometry and global transport in great detail, but individual transport components (e.g., direct reflections, inter-reflections, caus-tics, etc.) are coupled nontrivially in both space and time. To overcome this complexity, we observe that transient light trans