Intelligent Robotic Navigation Through Simultaneous Localization and Mapping with Convolutional Neural Networks

A study is presented on intelligent robotic navigation through simultaneous localization and mapping (SLAM) enhanced with convolutional neural networks (CNNs). The study included re-training a pre-trained CNN network for object detection, recognition and depth estimation in a laboratory setting, implementing a feature-based monocular SLAM algorithm (ORB-SLAM) within robot operating system (ROS) framework and integrating both the re-trained CNN and ORB-SLAM to intelligently guide a robot during navigation to reach target objects while avoiding obstacles. The visual SLAM (ORB-SLAM) enhanced with CNN for object detection, recognition and depth estimation was adapted and implemented in real-time. A Kobuki Turtlebot with Xbox 360 camera along with its on-board laptop (CPU based) was selected as the mobile robotic platform for the implementation within ROS framework. The proposed system successfully combined the capabilities of ORB-SLAM with CNN for real-time autonomous navigation of the robot in an enclosed environment. The power of edge computing with graphics processing unit (GPU) based hardware platform Jetson TX2 along with open-source software library TensorFlow suitable for implementation of deep learning architectures including CNN were utilized within ROS for real-time operation. The effectiveness of the system is illustrated through case studies that required the robot to avoid obstacles while locating designated objects within the map including a maze in a laboratory setting. In each case, the robot was able to plan a path towards the target objects using the map it saved from the ORB-SLAM-CNN implementation. The map was continuously updated accommodating changes in the environment while the robot was navigating towards the target objects. Details of algorithms developed, hardware and software support, real-time implementation, and results of different case studies are presented along with recommendations for future work