Multiview 3D Human Motion Tracking with Soft-Joint Constrained ICP

本論文的研究目的是要從相機所觀測到的影像序列中追蹤人體的姿態，並且沒有限定欲追蹤的姿態種類(如走路、跑步…等)，亦即被觀測者的動作不受到任何的限制。為了解決單一視角觀測容易發生自我遮蔽、與缺乏深度資訊產生姿勢估測模稜兩可的情形，我們透過多台攝影機取得多視角的影片，並建立三維人體容積，如此可有效地整合多視角的資訊。於人體的眾多關節具有極高的自由度，我們提出了一個階層式的人體姿態追蹤方法。在每個時間點先估測出軀幹姿態後，再進行四肢姿態的估測。我們採用廣泛被應用於高維度追蹤的粒子濾波器作為最難估測的軀幹姿態之追蹤方法，原因是粒子濾波器的好處在於能描述非線性及多極值的後測機率分布。然而對於階層式的人體姿態追蹤，其缺點在於軀幹姿態估測的正確性會連動地影響四肢的估測結果。為了降低軀幹誤估對四肢姿態估測的影響，我們採用結合了柔性關節規範之疊代最近點演算法。柔性關節規範允許四肢能脫離固定關節的局限，能在關節附近範圍移動，減少受軀幹姿態誤估的干擾。疊代最近點演算法則能將四肢使用柔性關節時需要的 7 個維度粒子濾波器，減少至只需決定肘部或膝部關節角度的 1 個自由度。對於四肢姿態追蹤，同時具備了粒子濾波器與柔性關節規範之疊代最近點演算法優點，使得我們的方法即使在四肢短時間內做出高速的動作時，仍能獲得有效的追蹤結果。們亦發現人體軀幹的方向與四肢的姿態具有相當之關連性，當我們知道四肢關節的位置時，通常就能預估出軀幹的姿態，尤其當我們擁有可靠的四肢動作資訊時。為了提高軀幹追蹤的正確性，我們藉由前一個時間點所估測之四肢柔性關節位置，進而預測目前時間點之軀幹姿態，使得粒子濾波器能有更可靠的估測依據。整合軀幹與四肢的姿態追蹤結果，我們提供了三維人體姿態追蹤問題一個有效的解決方法。In this thesis, we aim to track 3D human motions in image sequences captured from multiple cameras. The target motion is not limited to specific kinds of human motions, such as walking or jogging, that is, there is no restrictions imposed on possible human motions. Because self-occlusion and depth ambiguity occur easily when using only one single camera, we obtain multiple videos captured with multiple cameras from different viewpoints to reconstruct 3D shape volume of the target subject, which is an effective way to integrate information from multiple views.e propose a hierarchical human motion tracking method that can effectively capture human articulated motions with high degrees of freedom (DOFs). At each time step, the torso motion is estimated first and then the estimation of the limbs motions is carried out individually. The particle filtering, which is a popular method for high dimensional tracking, is adopted to track the torso motion because it can deal with the nonlinear and multimodal posterior probability distributions.ne disadvantage of hierarchical human motion tracking is that torso tracking errors may deteriorate limbs motion estimation. To reduce the interference from inaccurate torso motions, we propose a soft-joint constrained ICP (Iterative Closest Point) method to estimate limb motions. In contrast to hard joints, limbs with soft joints are allowed to move freely in a small range of area, so it is still possible to track limb motions even with inaccurate torso motions. However, the DOFs of each limb increase from 4 to 7 when the soft-joint constraint is used. The proposed soft-joint constrained ICP can efficiently determines 6 DOFs such that only 1 DOF (elbow/knee) is left for the particle filtering. Integrating the advantages of particle filtering and soft-joint constrained ICP at the same time, our method can effectively track limb motions even when there is large motion in a short period of time.oreover, we find that the torso motion is strongly related to the limbs motions. If the states of the four limbs are known, it is usually possible to predict the torso state without other information, especially when the limbs states are reliable. In order to improve torso motion tracking, the limbs motions estimated at the previous time step can provide reliable hypotheses of current torso state which is implemented as sampling particles from limbs states for torso tracking. We have conducted experiments with multiple video sequences of different motions, and the results show that our method is effective and reliable for 3D human motion tracking.口試委員會審定書 iii謝 v要 viibstract ixist of Figures xiii Introduction 1.1 Problem and Challenges 1.2 Proposed Method 3.3 Overview of Our Method 5.4 Outline of the Thesis 6 RelatedWorks 7.1 Model-Free vs. Model-Based 8.2 Single View vs. Multiple View 9.3 Image-Based Localization vs. Video-Based Tracking 11.3.1 Kalman Filtering vs. Particle Filtering 12.3.2 Advanced Particle Filtering 12.3.3 Hierarchical Particle Filtering 14 Model-Based 3D Human Motion Tracking 15.1 3D Human Model 15.1.1 Figure Parameters 16.1.2 Motion Parameters 18.2 3D Volume Reconstruction 20.2.1 Introduction to Volume-Based Visual Hull Construction 21.2.2 Implementation to Voxel-based Approach 23.3 Particle Filter Tracking 25.3.1 General Particle Filtering 25.3.2 Hierarchical Particle Filtering 29 Soft-Joint Constrained ICP and Torso Prediction 33.1 Introduction to ICP 34.2 Soft-Joint Constrained ICP 36.3 Voxel Labeling 41.4 Torso Prediction with Soft Joint Locations 44 Experiments 47 Conclusions and FutureWorks 59.1 Conclusions 59.2 Future Works 59ibliography 6