Vision-based Finger Detection and Its Applications

在本研究中，我們提出一項即時的手指偵測技術。此技術只需要將一個紅外線攝影機與紅外線照射燈，裝設在用來互動的平面之下，再利用攝影機取得的影像進行分析，就能即時地得到正確偵測的結果。此技術的主要貢獻如下：第一，我們只利用容易取得且價格便宜的的紅外線攝影機和和紅外線照射燈，就能夠進行手指的偵測。第二，針對光線分佈不平均的情況，這個偵測技術也能正確地執行。手指偵測可應用的層面相當廣泛，我們主要將此技術應用在兩個互動系統上，分別是i-m-Top以及3D魔幻水晶球(Magic Crystal Ball，或簡稱MaC Ball)。i-m-Top是一個使用背部投影的互動式多重解析桌面顯示系統。由於系統在投影時，需要散光片讓影像顯示在桌面上，而此技術能根據散光片的特性，正確地分辨出是手指接觸到桌面，或只是在桌面上盤旋兩種情形。我們亦利用這項手指偵測技術，針對i-m-Top設計了兩個應用。第一個應用是提供基本的文件編寫及瀏覽功能，第二個應用是提供一個影片檢索的系統。3D魔幻水晶球(MaC Ball)是一個互動式的3D顯示系統，我們結合了手指偵測及光學流動運動偵測，研發出一種有趣且直覺的操縱方式。此系統適合擺設在博物館、主題展示館、藝廊以及任何需要公共展示的地方，用以陳列各項重要或價值不菲的物品。成功塑造出虛擬展示空間，提供使用者更新的體驗。In this paper, we developed a real-time fingertip detection technique. The proposed technique works on images captured by infrared cameras. The cameras are settled on one side of the interactive surface, capturing human gestures taken place on the other side. The novelties of our technique are: firstly, the technique works with an infrared camera coupled with an infrared illuminator, which is easy to deploy and cost effective. Secondly, the proposed technique is designed to be robust for a non-uniform illuminated surface. We have deployed the technique on two applications: i-m-Top and Magic Crystal Ball (MaC Ball). i-m-Top is a rear-projection multi-resolution tabletop which provides several basic functions of browsing and modifying documents on the tabletop. With the characteristics of the projection diffuser, we can separate finger touching from palm hovering events when the user interacts with the diffuser. A video retrieval system working with finger touching and palm hovering interactions is presented on i-m-Top to show its intuitive operations. MaC Ball is an interactive display system that is suitable in museums, galleries, and exhibitions to demonstrate 3D treasures. For the interactions of MaC Ball, we integrate the fingertip finding technique and optic-flow motion detection to provide the users with natural interactions to experience the virtual exhibition.1 Introduction 1 2 Related Work 4 3 Finger Detection and Tracking 7 3.1 System Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2 Finding Fingertips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.3 Tracking Fingertips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4 Application I: i-m-Top 16 4.1 Design Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.2 Hardware Con‾guration . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.3 Software Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.3.1 Fingertips Detection . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.3.2 Palms Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.3.3 Association between Fingertips and Palms . . . . . . . . . . . . . 20 4.3.4 Pen Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.3.5 Priority of Pens and Palms . . . . . . . . . . . . . . . . . . . . . . 21 4.4 Some Applications to the i-m-Top . . . . . . . . . . . . . . . . . . . . . . 22 4.4.1 Digital Document Browsing and Editing . . . . . . . . . . . . . . 22 4.4.2 Video Retrieval System . . . . . . . . . . . . . . . . . . . . . . . . 23 5 Application II: Magic Crystal Ball 28 5.1 Design Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.2 Hardware Con‾guration . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5.3 Software Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.3.1 Motion Detection for Waving Gesture . . . . . . . . . . . . . . . . 33 5.3.2 Fingertip Finding for Pointing Gesture . . . . . . . . . . . . . . . 34 5.3.3 Pressure Sensing for Separating Touch and Hover Interactions . . 35 5.4 Some Applications to the MaC Ball: Virtual Exhibition . . . . . . . . . . 38 5.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.4.2 Virtual Museum . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.4.3 Relic Browsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6 Conclusion and Future Work 45 Bibliography4