Fisher III. A mixture of manhattan frames: Beyond the manhattan world

Objects and structures within man-made environments typically exhibit a high degree of organization in the form of orthogonal and parallel planes. Traditional approaches to scene representation exploit this phenomenon via the some-what restrictive assumption that every plane is perpendicu-lar to one of the axes of a single coordinate system. Known as the Manhattan-World model, this assumption is widely used in computer vision and robotics. The complexity of many real-world scenes, however, necessitates a more flex-ible model. We propose a novel probabilistic model that describes the world as a mixture of Manhattan frames: each frame defines a different orthogonal coordinate sys-tem. This results in a more expressive model that still ex-ploits the orthogonality constraints. We propose an adap-tive Markov-Chain Monte-Carlo sampling algorithm with Metropolis-Hastings split/merge moves that utilizes the ge-ometry of the unit sphere. We demonstrate the versatility of our Mixture-of-Manhattan-Frames model by describing complex scenes using depth images of indoor scenes as well as aerial-LiDAR measurements of an urban center. Addi-tionally, we show that the model lends itself to focal-length calibration of depth cameras and to plane segmentation. 1