Recognizing and Segmenting Objects in the Presence of Occlusion and Clutter

One of the fundamental problems of computer vision is to detect and localize objectssuch as humans and faces in images. Object detection is a building block for a widerange of applications including self-driving cars, robotics and face recognition. Thoughsignificant progress has been achieved in these tasks, it is still challenging to obtainrobust results in unconstrained images. Real world scenes usually contain more than oneobject and it is very likely that some parts of an object are occluded by other objectsin the scene. To tackle occlusion, image features generated by occlusion should be explicitlymodeled rather than treated as noise. In this thesis, a deformable part model for detectionand keypoint localization is introduced that explicitly models part occlusion. The proposedmodel structure makes it possible to augment positive training data with large numbers ofsynthetically occluded instances. This allows us to easily incorporate the statistics ofocclusion patterns in a discriminatively trained model. To exploit bottom-up cues such asoccluding contours and image segments, we extend the proposed model to utilize bottom-upclass-specific segmentation in order to jointly detect and segment out the foregroundpixels belonging to the object.In these approaches, a detector for a single object category is trained which operatesindependently of other detections in the scene. An appealing alternative approach fordetection in cluttered images is to move from single object detection to whole-imageparsing. The presence of occlusion can then be “explained away” by the presence of anoccluding object. We model multi-object detection by classifying each pixel of the image(semantic segmentation) using Convolutional Neural Network. CNN architectures have terrificrecognition performance but rely on spatial pooling which makes it difficult to adapt themto tasks that require dense, pixel-accurate labeling. We demonstrate that while theapparent spatial resolution of convolutional feature maps is low, the high-dimensional featurerepresentation contains significant sub-pixel localization information. We describe amulti-resolution reconstruction architecture based on a Laplacian pyramid that uses skipconnections from higher resolution feature maps and multiplicative gating to successivelyrefine segment boundaries reconstructed from lower-resolution maps. We demonstrate that this approach yields state-of-the-art semantic segmentation results without resorting to morecomplex random-field inference or instance detection driven architectures