Simultaneous Association and Localization for Multi-Camera Multi-Target Tracking

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·컴퓨터공학부, 2017. 8. 최진영.In this dissertation, we propose two approaches for three-dimensional (3D) localizing and tracking of multiple targets by using images from multiple cameras with overlapping views. The main challenge is to solve the 3D position estimation problem and the trajectory assignment problem simultaneously. However, most of the existing methods solve these problems independently. Unlike single camera multi-target tracking, it is much more complicated to solve both problems because the relationship between cameras is also taken into consideration in multi-camera. To tackle this challenge, we present two approaches: mixed multidimensional assignment approach and variational inference approach. In the mixed multidimensional assignment approach, we formulate the data association and 3D trajectory estimation problem as the mixed optimization problem with discrete and continuous variables and suggest an alternative optimization scheme which jointly solves the two coupled problems. To handle a large solution space, we develop an efficient optimization scheme that alternates between two coupled problems with a reasonable computational load. In this optimization formulation, we design a new cost function that describes 3D physical properties of each target. In the variational inference approach, we establish a maximum a posteriori (MAP) problem over trajectory assignments and 3D positions for given detections from multiple cameras. To find a solution, we develop an expectation-maximization scheme, where the probability distributions are designed by following the Boltzmann distribution of seven terms induced from multi-camera tracking settings.1 Introduction 1 1.1 Background & Challenges 1 1.2 Related Works 4 1.3 Problem Statements & Contributions 8 2 Mixed Multidimensional Assignment Approach 12 2.1 Problem Formulation 12 2.1.1 Problem Statements 12 2.1.2 Cost Design 17 2.2 Optimization 22 2.2.1 Spatio-temporal Data Association 23 2.2.2 3D Trajectory Estimation 31 2.2.3 Initialization 33 2.3 Application: Real-time 3D localizing and tracking system 35 2.3.1 System overview 36 2.3.2 Detection 37 2.3.3 Tracking 39 2.4 Appendix 42 2.4.1 Derivation of equation (2.35) 42 3 Variational Inference Approach 44 3.1 Problem Formulation 44 3.1.1 Notations 44 3.1.2 MAP formulation 46 3.2 Optimization 48 3.2.1 Posterior distribution 48 3.2.2 V-EM algorithm 51 3.3 Appendix 56 3.3.1 Derivation of equation (3.12) 56 3.3.2 Derivation of equation (3.27-3.32) 56 3.3.3 Deriving optimal mean and covariance matrix (3.33-3.35) 59 3.3.4 Definition of A and b in (3.22) 62 4 Experiments 63 4.1 Datasets 63 4.1.1 PETS 2009 63 4.1.2 PSN-University 64 4.2 Evaluation Metrics 66 4.3 Results and Discussion 67 4.3.1 Mixed Multidimensional Assignment Approach 67 4.3.2 Variational Inference Approach 82 4.3.3 Comparisons of Two Approaches 93 5 Conclusion 98 5.1 Concluding Remarks 98 5.2 Future Work 99 Abstract (In Korean) 112Docto