Color and depth sensing sensor technologies for robotics and machine vision

Robust scanning technologies that offer 3D view of the world in real time are critical for situational awareness and safe operation of robotic and autonomous systems. Color and depth sensing technologies play an important role in localization and navigation in unstructured environments. Most often, sensor technology must be able to deal with factors such as objects that have low textures or objects that are dynamic, soft, and deformable. Adding intelligence to the imaging system has great potential in simplifying some of the problems. This chapter discusses the important role of scanning technologies in the development of trusted autonomous systems for robotic and machine vision with an outlook for areas that need further research and development. We start with a review of sensor technologies for specific environments including autonomous systems, mining, medical, social, aerial, and marine robotics. Special focus is on the selection of a particular scanning technology to deal with constrained or unconstrained environments. Fundamentals, advantages, and limitations of color and depth (RGB-D) technologies such as stereo vision, time of flight, structured light, and shape from shadow are discussed in detail. Strategies to deal with lighting, color constancy, occlusions, scattering, haze, and multiple reflections are discussed. This chapter also introduces the latest developments in this area by discussing the potential of emerging technologies, such as dynamic vision and focus-induced photoluminescence