Mitigating Distortion to Enable 360° Computer Vision

For tasks on central-perspective images, convolutional neural networks have been a revolutionary innovation. However, their performance degrades as the amount of geometric image distortion increases. This limitation is particularly evident for 360° images. These images capture a 180° x 360° field of view by replacing the imaging plane with the concept of an imaging sphere. Because there is no isometric mapping from this spherical capture format to a planar image representation, all 360° images necessarily suffer from some degree of geometric image distortion, which manifests as local content deformation. This corruptive effect hinders the ability of these groundbreaking computer vision algorithms to enable 360° computer vision, resulting in a performance gap between networks applied to central-perspective images and those applied to spherical images. This dissertation seeks to better understand the impact that geometric distortion has on convolutional neural networks and to identify spherical image representations that can mitigate its effect. This work argues that there are three requisite properties of any general solution: distortion-mitigation, transferability, and scalability. Bridging the performance gap requires reducing distortion in the image representation, developing tools to directly apply central-perspective image algorithms to spherical data, and ensuring that these algorithms can efficiently process high resolution spherical images. Drawing insight from the field of cartography, the subdivided regular icosahedron is proposed as a low-distortion alternative to the commonly used equirectangular and cube map spherical image formats. To address the non-Euclidean nature of this representation, a generalization of the standard convolution operation is proposed to map the standard convolutional kernel grid to the structure of any spherical representation. Finally, a new representation is proposed. Derived from the icosahedron, it represents a spherical image as a set of square, oriented, planar pixel grids rendered tangent to the sphere at the center of each face of the icosahedron. These "tangent images" satisfy all three requisite properties, offering a promising, general solution to the spherical image problem.Doctor of Philosoph