Unsupervised learning of holistic 3D scene understanding

2019-02-123D scene understanding is a fundamental and important problem in computer vision. This task aims at estimating different 3D geometrical cues of the scene layout. The geometrical cues include depth, optical flow, object boundary, object movement, etc.. Despite decades of efforts devoted into this topic, it is still considered as an open problem. There are various applications of 3D estimation in the real world, such as robotics, augmented reality and autonomous driving. ❧ Depending on the types of geometrical cues, this problem can be classified as estimating static and dynamic geometrical information. Static 3D estimation includes depth estimation, surface normal estimation, object boundary estimation, etc. These geometrical cues can be estimated from a single image. Dynamic 3D information includes optical flow, odometry, 3D object motion, etc. These dynamic informations are embedded with temporal meanings and can only be estimated from consecutive frames or video sequences. The static and dynamic geometrical cues are highly correlated and can reinforce each other in the estimation. For example, a good depth estimation can help understand the 3D movement of objects in the scene; on the other hand, accurate optical flow estimation is a strong prior of rigid objects and can reinforce the object boundary estimation