Real-time Dynamic Hand Gesture Recognition Based on Finger Features Using a Depth Sensing Camera

在人機互動的領域中，人們一直想要找個方法來取代傳統的鍵盤和滑鼠，所以在這情況下就衍生了利用手勢來進行對機器的操作。而運用手勢辨識的概念不但經常在科技電影可以看到，也在具有多點觸控的智慧型手機和觸控板上成為流行，但是觸控式螢幕尺寸的大小限制將會影響到手勢辨識的準確性以及多元性，因此本論文目的為利用三維空間資訊為主來達到即時的手勢辨識，且在無多點觸控能力之螢幕的情況下，依舊能夠辨識出使用者所作的手勢。 本系統使用Kinect感應器得到完整的三維資訊，並運用深度直方圖機制，無論在任何環境下都可以偵測出使用者的手，在使用K-means分群法下，即使手有重疊的情況也可以正確地區分數量。為了發展更多元的手勢，我們利用多指的合併和分開來發展更多元的手勢，但因為每個人的習慣和手指的粗細不盡相同，因此我們利用了機器學習和支持向量機依照不同的特徵值來判斷手指正確的數量，最後再利用有限狀態機來判斷動態的手勢。In recent years, people have tried to find more efficient ways to replace the old-fashioned keyboards and mice in communication between humans and computers. Among several attempts in this direction, gestures have received considerable attention as they already serve as a natural form of human interaction. The use of gestures in human-computer interaction, once only appeared in science fiction movies, has gradually become reality thanks to the advance of technologies such as multi-touch screens. The size of a touch screen, however, restricts the development of gesture recognition to a certain extent. The objective of this thesis is to develop a real-time system capable of recognizing hand gestures with a touch-less interface by taking advantage of 3D sensing capabilities of depth information. The proposed system acquires accurate 3D data from Kinect, and use depth histograms in order to perform hand localization from any arbitrary background. The K-means clustering algorithm is used to determine the number of hands found in the image, even when occlusion occurs due to hand overlapping. In order to accommodate a diversity of gestures, we take advantage of different combinations and separations of fingertips. To cope with a variety of user habits and thickness of fingers, we use machine learning and SVM to determine the accurate amounts of fingers based on different features. Finally, a finite-state machine is used to determine the dynamic gestures of hand movements.口試委員會審定書 # 誌謝 iii 目錄 iv 圖目錄 vi 表目錄 viii Chapter 1 緒論 1 1.1 動機 1 1.2 研究貢獻 2 1.3 組織架構 3 Chapter 2 相關研究 4 2.1 手勢操作起源 4 2.2 手勢辨識 5 2.2.1 姿勢行為分類 5 2.2.2 應用範圍 6 2.3 三維深度攝影機 10 2.3.1 立體視覺法 10 2.3.2 時差測距法 12 2.3.3 光編碼 14 2.4 感應器的選擇 16 Chapter 3 手勢辨識系統 18 3.1 手勢辨識系統流程介紹 18 3.2 手部定位 19 3.2.1 建立深度直方圖 19 3.2.2 K-means分群演算法 21 3.3 手部特徵擷取 22 3.3.1 凸包偵測 22 3.3.2 掌心與手腕 23 3.4 手指數量判別與指尖位置偵測 25 3.4.1 手指特徵 25 3.4.2 利用支持向量機判斷手指數量 27 3.4.3 指尖定位 31 3.5 手勢判斷 32 3.5.1 手勢分類 32 3.5.2 利用有限狀態機來判別手勢 35 Chapter 4 實驗結果與分析 37 4.1 分析手部定位 37 4.2 分析手指偵測 41 4.3 手勢辨識系統 45 4.4 系統效能分析 47 Chapter 5 結論與未來發展 48 REFERENCES 49 i 中文摘要 ii ABSTRAC