Real Time Panoramic Depth Imaging Using Standard Cameras

A computer vision is a special kind of scientific challenge as we are all users of our own vision systems. Our vision is definitely a source of the major part of information we acquire and process each second. A stereo vision is perhaps even greater challenge, since our own vision system is a stereo one and it performs a complex task, which supplies us with 3D information on our surroundings in a very effective way. Making machines see is a difficult problem. On one side we have psychological aspects of human visual perception, which try to explain how the visual information is processed in the human brain. On the other side we have technical solutions, which try to imitate human vision. Normally, it all starts with capturing digital images that store the basic information about the scene in a similar way that humans see. But this information represents only the beginning of a difficult process. By itself it does not reveal the information about the objects on the scene, their color, distances etc. to the machine. For humans, visual recognition is an easy task, but the human brain processing methods are still a mistery to us. One part of the human visual perception is estimating the distances to the objects on the scene. This information is also needed by robots if we want them to be completely autonomous. In this dissertation we present a stereo panoramic depth imaging system. The basic system is mosaic-based, which means that we use a single standard rotating camera and assemble the captured images in a multiperspective panoramic image. Due to a setoff of the camera's optical center from the rotational center of the system we are able to capture the motion parallax effect, which enables the stereo reconstruction. The camera is rotating on a circular path with the step defined by an angle equivalent to one-pixel column of the captured image. To find the corresponding points on a stereo pair of panoramic images the epipolar geometry needs to be determined. It can be shown that the epipolar geometry is very simple if we are doing the reconstruction based on a symmetric pair of stereo panoramic images. We get a symmetric pair of stereo panoramic images when we take symmetric columns on the left and on the right side from the captured image center column. This system however cannot generate panoramic stereo pair in real time. That is why we have suggested a real time extension of the system, based on simultaneously using many standard cameras. We have not physically built the real time sensor, but we have performed simulations to establish the quality of results. Both systems have been exhaustively analysed and compared. The analyses revealed a number of interesting properties of the systems. According to the basic system accuracy we definitely can use the system for autonomous robot localisation and navigation tasks. The assumptions made in the real time extension of the basic system have been proved to be correct, but the accuracy of the new sensor generally deteriorates in comparison to the basic sensor. Generally speaking, the dissertation can serve as a guide for panoramic depth imaging sensor design and related issues