Development of a Hybrid Haptic Feedback Virtual Assembly System with Natural User Interface

虛擬實境於工程組裝的應用近年來被廣泛的討論與實現，相較於在真實世界中進行組裝，使用者在虛擬環境中組裝可以達到降低操作成本、操作危險性，以及重複操作等好處。近年來由於科技技術的提升，許多新興的人機界面技術相繼誕生，因此，如何運用這些技術，來使虛擬組裝能更符合使用者日常生活中的操作習慣，邁向自然使用者界面(NUI)的設計，便成為本研究致力達成的目標。本研究提出一個具有擬真操作模式的虛擬組裝系統，供使用者進行直接運用雙手和使用工具的組裝，並擁有兩種力回饋模式，提供使用者在視覺之外，更真實的觸覺感受。本研究使用Chang(2015)所研發的震動力回饋手套提供使用者觸覺回饋，並搭配Leap Motion的手部位置偵測作為手部模擬，使用PHANToM OmniR提供使用者肢覺回饋作為組裝工具模擬。雙手應用和使用3D滑鼠來轉換視角的設計讓使用者在虛擬環境中能有更擬真的感受。最後，本研究設計了二組組裝流程進行使用者測試，結果顯示使用者傾向於用震動力回饋手套抓取大物件，用PHANToM OmniR抓取小物件；在組裝效率方面，震動力回饋手套加PHANToM OmniR的複合模式得分最高，證實了本研究所建立的虛擬組裝系統具有組裝效率並且在人機介面的設計上符合自然使用者界面。Applications of assembly in virtual reality are increasingly discussed and implemented in recent years. Compared to real world assembly, users can complete necessary tasks in virtual reality with lowered cost, less potential hazards, and the ability of repeat manipulation. Thanks to the development of technologies, many human-computer interfaces with novel technologies have been born. Therefore, how to manage and combine these technologies into a virtual assembly system and also design it according to user’s daily habits, trying to achieve the design of natural user interface becoming a goal for this research. A virtual assembly system with realistic operating modes were proposed by us, which can provide users directly using their hands or using assembly tools to assemble, and it also equipped two different haptic feedbacks providing users with not only visual feedback but also haptic feedback. A pair of gloves developed by Chang(2015) were tracked by Leap Motion were served as users hands simulation providing users with vibrio-tactile feedback, and the PHANToM OmniR was used to provide kinesthetic force feedback to simulate assembly tools in VR. In order to provide users with a more realistic VR environment, Dual-handed manipulation was designed and 3D mouse was used for changing viewpoints. At the end, two case studies were designed. The result shows that users tend to use gloves to grab large objects, and use PHANToM OmniR to grab small objects. Considering of the task completion time, the hybrid mode, which combined gloves and PHANToM OmniR, was the most efficient mode. As a result, the virtual assembly system with hybrid mode developed by us was confirmed to be efficient and has natural user interface