Closed-Form Solution to the Structure from Motion Problem by Fusing Visual and Inertial Sensing

The structure from motion problem (SfM) consists in determining the three-dimensional structure of the scene by using the measurements provided by one or more sensors over time (e.g. vision sensors, ego-motion sensors, range sensors). Solving this problem consists in simultaneously performing self motion perception (sMP) and depth perception (DP). In the case of visual measurements only, the SfM has been solved up to a scale \cite{Chi02,Dav07,Har97,Lon81,Nis04} and a closed form solution has also been derived \cite{Har97,Lon81,Nis04}, allowing the determination of the three-dimensional structure of the scene, without the need for any prior knowledge. The case of inertial and visual measurements has particular interest and has been investigated by many disciplines, both in the framework of computer science \cite{Bry08,Jon11,Kelly11,Stre04} and in the framework of neuroscience (the visual-vestibular integration for sMP \cite{Bert75,Fets10,Mac08,Zup02} and for DP \cite{Dokka11}). Prior work has answered the question of which are the observable modes, i.e. the states that can be determined by fusing visual and inertial measurements \cite{Bry08,Jon11,Kelly11,INRIA11,TRO12}. The questions of how to compute these states in the absence of a prior, and of how many solutions are possible, have only been answered very recently \cite{INRIA11,TRO12}. Here we derive a very simple and intuitive derivation of the solution introduced in \cite{INRIA11,TRO12}. We show that the SfM problem can have a unique solution or two distinct solutions or infinite solutions depending on the trajectory, on the number of point-features and on the number of monocular images where the same point-features are seen. Our results are relevant in all the applications which need to solve the SfM problem with low-cost sensors and which do not demand any infrastructure. Additionally, our results could play an important role in neuroscience by providing a new insight on the process of vestibular and visual integration