Human Robot Interaction with Vision-Based Simultaneous Localization and Mapping for Service Robotics

隨著科技發展，自動化產品已經成為人們生活中不可或缺的一部分，伴隨著高齡化時代的來臨，機器人會漸漸地進入人們的日常生活中，智慧型服務機器人便是為此目的而開發。服務型機器人必須要有在環境中移動與自我定位的能力，以達到與環境或人們互動的應用。 本論文的目的在於設計一個利用視覺特徵點作為位置參考的同步定位與建置地圖並同時結合人機互動功能的系統，可以藉由與人互動的同時記錄背景的視覺特徵點，並加以處理作為定位的參考，達到機器人自主化的目的。論文中的實驗主要分為兩大部分，第一部分主要是驗證當機器人在進行同步定位與建置地圖時，若有人進入機器人的視覺範圍中，會對定位的結果會造成怎樣的影響。第二部分則是將第一部分驗證的結果結合人機互動，使得機器人可以一邊跟隨使用者，一邊進行環境的探索，建構地圖。 本論文利用FAST演算法找出特徵點位置，使用雙眼立體視覺與Kinect獲得視覺特徵點之立體座標，並提出特徵點緩衝器的概念濾除不穩定的特徵點，以減輕系統運算的負擔，另外也提出了特徵點相位的想法，用來區分聚集在一起的特徵點，使得資料匹配有較明顯的唯一性。在人體偵測方面採用了方向梯度直方圖(Histogram of oriented gradient)與Kinect的深度資料找出人的位置並跟隨人移動，同時並濾除位於人體身上的特徵點，降低定位誤差與加快運算速度。最後利用拓展型卡曼濾波器(Extended Kalman Filter)來融合感應器與制動器的誤差，推算出機器人與地標可能的所在位置並繪出環境地圖。本論文所提出的軟體架構與設計模式都是以標準 C/C++ 程式語言，結合 MRPT (The mobile robot programming toolkit)與OPENNI函式庫於Visual Studio軟體平台中進行開發。With development of technology, automation products have been an indispensable part in human’s daily lives. Along with the aging era, robot will come into our daily lives gradually. Intelligent service robot is developed for this purpose. Service robot should have ability to move and self-localization in unknown environment, so that it would achieve the application to interact with human or its surrounding. The purpose of this thesis is to develop a system which uses visual landmarks as position reference to locate itself and build the surrounding map simultaneously. The system also consists of human robot interaction aspect. Through interacting with human, the robot records visual landmarks and locate itself to satisfy autonomy demand. The experiments are separated into two main parts in this thesis. The first part is designed to verify if human enter the visual range of robot during executing V-SLAM, what will be the impact on result. The second part is combining the result of first part and human robot interaction together. Let robot follow people and construct the map of its surrounding simultaneously. This thesis employs FAST corner detector to extract the location of feature points and obtains 3D coordinates by stereo vision and Kinect. We propose a concept named “feature buffer”, and the feature buffer would filter out temporary feature points, and it reduces computational loading of system. Besides, we also propose an idea about “feature orientation”. Feature orientation would distinguish the landmarks clustering together and enhance the uniqueness of landmarks, so that we would get better result of data association. In this thesis the Histogram of oriented gradient (HOG) and the depth data of Kinect are used as human detector. The human detector finds out the location of human, and the system uses this information to remove the feature points on human body and also employs this information to follow people. Finally, we use the Extended Kalman Filter (EKF) to fuse the error of sensor and odometer. After fusing, the possible locations of robot and landmarks are calculated, and the map of the environment is constructed. In this thesis all the systems models and software frameworks are implemented with C/C++ programming language, MRPT (The mobile robot programming toolkit), and OPENNI library, and all of them are integrated and developed in Visual Studio