[[alternative]]Hand gesture recognition based messenger system

碩士[[abstract]]本論文研製一個基於手勢輸入之即時通系統。此系統利用撓性感測器(flex sensor)與加速度計，搭配一組穿戴式機構，量測各手指的彎曲角度和手腕的傾斜方向，並將量測所得資訊以RS232傳輸協定傳至電腦端，經濾波後進行手勢辨識演算以產生相對應的文字。最後搭配TCP/IP協定，以自製之即時通介面與其他電腦進行互動。 在手勢辨識演算方面，本論文先採用一決策樹架構，將不同手勢進行初步分類，再由一改良式的機率類神經網路進行最後的手勢辨識。實驗顯示此方法降低了辨識時的計算負擔，提高了整體的辨識效率。[[abstract]]This thesis develops a messenger system based on hand gesture recognition technique. Flex sensors and an accelerometer combined with a wearable gesture sensing device are used to measure the bending angle of each finger and the wrist. The measurements are transmitted to PC by RS232 transmission protocol. After filtering to smooth the data, these data are processed through the gesture recognition algorithm to generate the corresponding text. Finally, the users can interact with the others by the homemade messenger system interface through TCP / IP protocol. The proposed gesture recognition algorithm consists of two major steps, which first utilizes the concept of decision tree to classify the filtered measured data of different gestures, and then performs more delicate gesture recognition by a probability neural network to reach a verification result. Experiment shows that an improved verification rate with reduced computational burden of the problem is obtained by the proposed method.[[tableofcontents]]目 錄 中文摘要.................................................................................................I 英文摘要................................................................................................II 目錄.......................................................................................................III 圖目錄...................................................................................................VI 表目錄....................................................................................................X 第1章 緒論....................................................................................................1 1.1 研究動機....................................................................................................1 1.2 研究背景....................................................................................................3 1.3 論文架構....................................................................................................6 第2章 背景知識....................................................................................................7 2.1 感測元件....................................................................................................7 2.1.1 撓性感測器....................................................................................................7 2.1.2 三軸加速度計....................................................................................................9 2.2 MSP430微控制器....................................................................................................14 2.3 通訊協定....................................................................................................16 2.3.1 I2C 同步串列通訊....................................................................................................16 2.3.2 RS232傳輸協定....................................................................................................19 2.3.3 TCP/IP傳輸協定....................................................................................................21 2.4 SQL資料庫.................................................................................................... 22 第3章 具有辨識能力之穿戴式手部裝置....................................................................................................24 3.1 系統設計與架構....................................................................................................24 3.2 穿戴式手勢感測裝置之設計....................................................................................................28 3.2.1 flex sensor撓性感測器與放大電路....................................................................................................30 3.2.2 加速度計模組....................................................................................................33 3.2.3 以MSP430為核心之訊號轉換與傳輸處理模組....................................................................................................34 3.2.4 平均移動濾波器....................................................................................................37 3.3 手勢辨識演算法則....................................................................................................39 3.3.1 決策樹....................................................................................................41 3.3.2 機率類神經網路....................................................................................................43 第4章 即時通訊互動介面....................................................................................................49 4.1 通訊架構....................................................................................................49 4.2 即時通介面功能....................................................................................................57 4.2.1 個人化功能介紹....................................................................................................57 4.2.2 SQL資料庫設定....................................................................................................63 第5章 實驗結果....................................................................................................69 5.1 實驗模擬環境介紹....................................................................................................69 5.2 實驗結果....................................................................................................73 5.2.1 本文方法的辨識結果....................................................................................................73 5.2.2 三種方法的比較....................................................................................................76 第6章 結論與未來研究方向....................................................................................................80 6.1 結論....................................................................................................80 6.2 未來研究方向....................................................................................................81 參考文獻……………………………………………………..…...….82 圖目錄 圖 2.1 撓性感測器....................................................................................................7 圖 2.2 撓性感測器彎曲情況....................................................................................................8 圖 2.3 flex sensor電阻改變原理示意圖....................................................................................................8 圖 2.4 使用次數與總電阻變化量關係圖....................................................................................................9 圖 2.5 壓阻式加速度計結構運作圖....................................................................................................10 圖 2.6 電容式加速度計結構運作圖....................................................................................................11 圖 2.7 LSM303DLH電容式三軸加速度計架構圖....................................................................................................12 圖 2.8 加速度計動作前後圖....................................................................................................13 圖 2.9 MSP430之馮紐曼架構....................................................................................................15 圖 2.10 MSP430F1611內部架構圖....................................................................................................15 圖 2.11 I2C連線示意圖....................................................................................................17 圖 2.12 完整I2C資料轉換....................................................................................................18 圖 2.13 串列傳輸示意圖....................................................................................................19 圖 2.14 RS232傳輸格式....................................................................................................20 圖 2.15 TCP/IP網路模型....................................................................................................21 圖 2.16 TCP/IP封包格式....................................................................................................22 圖 3.1 使用情境示意圖....................................................................................................25 圖 3.2 本研究之系統架構圖....................................................................................................27 圖 3.3 穿戴式手勢感測裝置實體圖....................................................................................................28 圖 3.4 穿戴式手勢感測裝置的運作流程....................................................................................................29 圖 3.5 撓性感測器安裝情形....................................................................................................31 圖 3.6 撓性感測器放大電路....................................................................................................32 圖 3.7 感測器I2C接線方式....................................................................................................33 圖 3.8 RS232傳輸封包格式....................................................................................................35 圖 3.9 處理模組的運作狀態流程....................................................................................................36 圖 3.10 移動平均濾波器視窗移動....................................................................................................37 圖 3.11 濾波處理前後比較圖....................................................................................................38 圖 3.12 ASL美國手語示意圖....................................................................................................39 圖 3.13 決策樹判斷條件對照表....................................................................................................41 圖 3.14 決策樹架構圖....................................................................................................42 圖 3.15 傳統機率類神經網路的架構....................................................................................................43 圖 3.16 改良式機率類神經網路的架構....................................................................................................46 圖 4.1 本文即時通系統的網路架構....................................................................................................50 圖 4.2 連線狀態時客戶端的運作狀態流程圖....................................................................................................52 圖 4.3 連線狀態時伺服端的運作狀態流程圖....................................................................................................54 圖 4.4 伺服端在聊天時的運作狀態流程圖....................................................................................................56 圖 4.5 本文即時通介面....................................................................................................58 圖 4.6 通訊錄介面....................................................................................................59 圖 4.7 通訊錄聯絡人基本資料....................................................................................................59 圖 4.8 對話記錄記事本檔案....................................................................................................60 圖 4.9 對話記錄介面....................................................................................................61 圖 4.10 提示功能介面....................................................................................................62 圖 4.11 圖形化管理工具SSMSE....................................................................................................64 圖 4.12 SQL查詢語言的輸入....................................................................................................64 圖 4.13 建立資料庫....................................................................................................65 圖 4.14 建立資料表....................................................................................................67 圖 4.15 觀察資料表....................................................................................................67 圖 5.1 實驗環境....................................................................................................69 圖 5.2 實驗模擬介面....................................................................................................70 圖 5.3 SQL資料庫所儲存之訓練樣本....................................................................................................71 圖 5.4 儲存在Excel檔案之訓練樣本....................................................................................................72 圖 5.5 SQL Sever匯出匯入精靈....................................................................................................72 圖 5.6 實驗1之實驗模擬介面....................................................................................................73 圖 5.7 Y手勢於本文之類神經網路的機率密度函數....................................................................................................75 圖 5.8 實驗2之實驗模擬介面....................................................................................................76 圖 5.9 相似手勢之舉例....................................................................................................77 圖 5.10 本文之類神經網路的機率密度函數....................................................................................................78 圖 5.11 傳統類神經網路的機率密度函數....................................................................................................79 表目錄 表 3.1 不同方法之神經元總數與遞減率....................................................................................................48 表 5.1 本文方法之辨識結果....................................................................................................74 表 5.2 10次測試中不同方法的錯誤次數與辨識率....................................................................................................77[[note]]學號: 600460199, 學年度: 10