SimCrane 3D+: A Crane Simulator with Kinesthetic and Stereoscopic Vision

在營建工業，多用途的特性使吊車在工地扮演著關鍵的角色，提供適當的吊車操作訓練對吊車工安及良好的吊車施工品質是非常重要的。近年來，研究者開發了吊車模擬器用於吊車操作訓練。吊車模擬器的使用可降低吊車訓練的成本，包括機具及場地的租用；另一方面，吊車模擬器可提供擬真的工地環境和工項來增進操作者熟練度。然而，目前吊車模擬器的視覺資訊顯示方式無法提供足夠的視覺深度訊息。若吊車操作者無法透過雙眼及身體位置得到相對應的正確三維資訊，則訓練的效果將會大打折扣，尤其是在模擬複雜的空間場景時；然而實際工地場景通常是複雜的，擬真場景是使用吊車模擬器的重要目的之一。因此，本研究增進了兩個機制於吊車模擬系統：體感視覺與立體視覺。為了實作體感視覺機制，本研究使用微軟公司(Microsoft)的Kinect作為身體資訊的偵測器，並推導了兩個轉換矩陣(transformation matrices)用於三維電腦繪圖流程(rendering pipeline)：一個是用來將從Kinect讀取到的視點位置導入繪圖流程中；另一個是用於計算視角對三維繪圖畫面的影響。另外，本研究使用NVIDIA 3D VisionTM套裝工具實現立體視覺，包含了立體視覺的繪圖演算法及相關硬體設備，並搭配了3D眼鏡及3D電視。綜合了體感視覺與立體視覺，本研究開發了體感立體視覺吊車模擬系統(SimCrane3D+)，此系統是使用微軟公司的XNA遊戲開發工具開發，本研究也建立了虛擬的吊裝工程場景作為研究案例。案例顯示，SimCrane3D+可藉由讀取操作者身體位置而提供對應其視角的連續動態畫面，及提供雙眼視覺的立體畫面。透過體感及立體視覺，操作者可感受到較佳的視覺深度訊息；而且當視角不同時，也會感受到相對應的視野及遮障。本研究成果未來可應用於吊車操作的虛擬訓練及精密吊裝工程的施工前虛擬演練。Since cranes have a critical and versatile role in construction sites, it is extremely important that operators be provided with adequate training to enable them perform erections safely and efficiently. Several researchers have developed crane simulators to facilitate operator training. The use of simulators reduces the costs associated with renting actual cranes, and enables the training of operators in a range of tasks and environments, allowing the use of virtual environments to develop the operators’ skills. However, a critical drawback common to most existing simulators is the lack of perception of depth. The effectiveness of training may be reduced if the 3D perspective, obtained through human eyes and body movements is not simulated, especially for complex erection tasks, which are the main objectives of virtual trainings. Therefore, this research added two major components to the simulation system, kinesthetic vision and stereoscopic vision. To realize kinesthetic vision, we integrated Microsoft Kinect as the motion sensor. We also derived two transformation matrices: one for the dynamic eye position captured by the motion sensor and the other for compensating for distortion induced due to the inclined view angle. Stereoscopic vision was realized by integrating the NVIDIA 3D VisionTM package, which includes a 3D rendering pipeline, a pair of 3D glasses, and a 3D display. We also developed a crane simulator, called SimCrane 3D+, by integrating kinesthetic vision and stereoscopic vision into a game framework based on the Microsoft XNA toolset. We also developed a typical erection scenario in a complex simulated environment as an example. We found that SimCrane 3D+ can process continuous readings from the motion sensor and smoothly render stereoscopic views. With the addition of kinesthetic and stereoscopic vision, users now have better depth perception and excellent visibility during the operation. The research results show that the system has great potential for training operators and rehearsing critical erections