Robust structure-based autonomous color learning on a mobile robot

textMobile robots are increasingly finding application in fields as diverse as medicine, surveillance and navigation. In order to operate in the real world, robots are primarily dependent on sensory information but the ability to accurately sense the real world is still missing. Though visual input in the form of color images from a camera is a rich source of information for mobile robots, until recently most people have focussed their attention on other sensors such as laser, sonar and tactile sensors. There are several reasons for this reliance on other relatively low-bandwidth sensors. Most sophisticated vision algorithms require substantial computational (and memory) resources and assume a stationary or slow moving camera, while many mobile robot systems and embedded systems are characterized by rapid camera motion and real-time operation within constrained computational resources. In addition, color cameras require time-consuming manual color calibration, which is sensitive to illumination changes, while mobile robots typically need to be deployed in a short period of time and often go into places with changing illumination. It is commonly asserted that in order to achieve autonomous behavior, an agent must learn to deal with unexpected environmental conditions. However, for true extended autonomy, an agent must be able to recognize when to abandon its current model in favor of learning a new one, how to learn in its current situation, and also what features or representation to learn. This thesis is a fully implemented example of such autonomy in the context of color learning and segmentation, which primarily leverages the fact that many mobile robot applications involve a structured environment consisting of objects of unique shape(s) and color(s) - information that can be exploited to overcome the challenges mentioned above. The main contributions of this thesis are as follows. First, the thesis presents a hybrid color representation that enables color learning both within constrained lab settings and in un-engineered indoor corridors, i.e. it enables the robot to decide what to learn. The second main contribution of the thesis is to enable a mobile robot to exploit the known structure of its environment to significantly reduce human involvement in the color calibration process. The known positions, shapes and color labels of the objects of interest are used by the robot to autonomously plan an action sequence to facilitate learning, i.e. it decides how to learn. The third main contribution is a novel representation for illumination, which enables the robot to detect and adapt smoothly to a range of illumination changes, without any prior knowledge of the different illuminations, i.e. the robot figures out when to learn. Fourth, as a means of testing the proposed algorithms, the thesis provides a real-time mobile robot vision system, which performs color segmentation, object recognition and line detection in the presence of rapid camera motion. In addition, a practical comparison is performed of the color spaces for robot vision – YCbCr, RGB and LAB are considered. The baseline system initially requires manual color calibration and constant illumination, but with the proposed innovations, it provides a self-contained mobile robot vision system that enables a robot to exploit the inherent structure and plan a motion sequence for learning the desired colors, and to detect and adapt to illumination changes, with minimal human supervision.Electrical and Computer Engineerin