Images and depth for high resolution, low-latency sensing and security applications

The thesis focuses on using images and depths for high resolution, low latency sensing, and then using these sensing techniques to build security applications. First, we introduce the usefulness of high quality depth sensing, and the difficulty to acquire such depth stream via pure hardware approach. Then, we propose our sensor fusion approach, which combines depth camera and color camera. Chapter 2 puts forward a low cost approach to use a high spatial resolution color stream to help aggressively increase the spatial resolution of the depth stream. Continuing this direction, Chapter 3 proposes to use optical ow to forward warp the depth stream according to a high frequency, low latency CMOS color stream. The warping can create a high frequency, low latency depth stream. In both Chapter 2 and Chapter 3, we show that the improved depth sensing can benefit lots of applications. In Chapter 4, we propose a SafetyNet, which can reliably detecting and rejecting adversarial examples. With the revolutionary SafetyNet architecture and the advanced depth sensing, we can reliably prove to users whether a picture of a scene is real or not. In sum, the thesis focuses on improving sensing technologies and building vision and security applications around the sensing technologies