A theoretical analysis of Bayes-optimal multi-target tracking and labelling

In multi-target tracking (MTT), we are often interested not only in finding the position of the multiple objects, but also allowing individual objects to be uniquely identified with the passage of time, by placing a label on each track. While there are many MTT algorithms that produce uniquely identified tracks as output, most of them make use of certain heuristics and/or unrealistic assumptions that makes the global result suboptimal of Bayesian sense. An innovative way of performing MTT is the so-called joint multi-target tracking, where the raw output of the algorithm, rather than being already the collection of output tracks, is a multi-target density calculated by approximating the Bayesian recursion that considers the entire system to have a single multidimensional state. The raw output, i.e. the calculated multi-target density, is thereafter processed to obtain output tracks to be displayed to the operator. This elegant approach, at least in theory, would allow us to precisely represent multi-target statistics. However, most joint MTT methods in the literature handle the problem of track labelling in an ad-hoc, i.e. non-Bayesian manner. A number of methods, however, have suggested that the multi-target density, calculated using the Bayesian recursion, should contain information not only about the location of the individual objects but also their identities. This approach, that we refer as joint MTTL (joint multi-target tracking and labelling), looks intuitively advantageous. It would allow us, at least in theory, to obtain an output consisting of labelled tracks that is optimal in Bayesian sense. Moreover, it would allow us to have statistical information about the assigned labels; for instance, we would know what is the probability that track swap may have occurred after some approximation of targets (or, in simpler words, we would know how much we can believe that a target is what the display says that it is). However, the methods proposed in the still emerging joint MTTL literature do not address some problems that may considerably reduce the usefulness of the approach. These problems include: track coalescence after targets move closely to each other, gradual loss of ambiguity information when particle filters or multiple hypotheses approaches are used, and dealing with unknown/varying number of targets. As we are going to see, each of the previously proposed methods handles only a subset of these problems. Moreover, while obtaining a Bayes-optimal output of labelled tracks is one of the main motivations for joint MTTL, how such output should be obtained is a matter of debate. This work will tackle the joint MTTL problem together with a companion memorandum. In this work, we look at the problem from a theoretical perspective, i.e. we aim to provide an accurate and algorithm-independent picture of the aforementioned problems. An algorithm that actually handles these problems will be proposed in the companion memorandum. As one of the contributions of the memorandum, we clearly characterize the so-called "mixed labelling" phenomenon that leads to track coalescence and other problems, and we verify that, unlike implied in previous literature, it is a physical phenomenon inherent of the MTTL problem rather than specific to a particular approach. We also show how mixed labelling leads to nontrivial issues in practical implementations of joint MTTL. As another of the contributions of the memorandum, we propose a conceptual, algorithm-independent track extraction method for joint MTTL estimators, that gives an output with clear physical interpretation for the user