To my family iii Acknowledgements My work over the last few years would not have been possible without the efforts of my PhD advisors. I would like to thank Rob Fergus and David Sontag for the many hours they spent providing insight, advice and direction. Thanks to Rob for taking me on as a student, helping me forge a path through 3D scene understanding and teaching me how to approach problems in research. Thanks to David for his patience and his unrelenting ability to see problems from different angles and propose creative solutions. I would also like to thank Lakshmi Subramanian who opened the door to my academic career and made the choice of starting a PhD an easier one. Many thanks to my collaborators. In particular, thanks to Derek Hoi...