A Hardware Implementation of Edge Detection and Lens Correction for 360-degree Panorama

Traditionally, image stitching allows users to combine multiple regular-sized images into a single wide-angle picture, often be named as a panoramic picture. In order to create such a panoramic picture, users usually need to take a lot of photographs, then upload them to a PC and stitch. Now, we only take one shot using an omnidirectional sensor, then we can get a 360-degree full environmental image. In this way, we can have a panoramic image easier and faster than before. Because of this reason, more and more researches and applications start to change traditional sensor to omnidirectional sensor. In recent year, mobile robot is an active research project, especially for robotic vision system. Eyes are the window of the soul. Hence, we believe that vision for mobile robot is as important as eye for human. Through omnidirectional sensor, mobile robots can receive more environmental information and then it will have more abilities for moving and locating. In this work, we will describe how to unwrap an omnidirectional image to a correctly panoramic image and use edge detection to get the whole environmental edge's information. Beside, we will implement the design into hardware to have a better computation performance. The experimental result will show out panoramic video output from an omnidirectional RGB camera input.Abstract i 1 Introduction 1 2 Related Work 4 2.1 Omnidirectional Vision System . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3 Unwrap 360-degree Omnidirectional Image and Edge Detection 10 3.1 Unwrapping 360-degree Omnidirectional Image . . . . . . . . . . . . . . . . . . 11 3.2 Sobel Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4 Hardware Implementation 16 4.1 Hardware Design Platform . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.1.1 Virtex-4 ML402 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.1.2 Video I/O Daughter Card . . . . . . . . . . . . . . . . . . . . . . 17 4.1.3 LVDS RGB Camera . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.1.4 Hardware Platform Diagram . . . . . . . . . . . . . . . . . . . . . 18 4.2 Hardware Implementation of Lens Correction for 360-degree Panorama . . . . . 19 4.3 Hardware Implementation of Edge Detection . . . . . . . . . . . . . . . . 21 4.4 Pipeline Flow of Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.5 Xilinx Design Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5 Experimental Results 25 6 Conclusion 3