Tracking by Planning

We introduce a method for tracking multiple people in acluttered street scene. We use global context to address the challenge oflong occlusion by endowing each tracked object with a planning agent.This planner uses context of the street scene, people and other movingobjects to reason about pedestrian intended behavior for tracking underocclusion and ambiguity. We extract short but robust trajectories called tracklets by tracking peoplewith a simple appearance model. We formulate the tracking problemas a batch mode optimization, linking tracklets into paths, each withsupporting evidence by an agent’s goal directed behavior, and imagepartial matching along the trajectory gap. We propose a global criteriafor consistent linking of the tracklet with planning that can correct localambiguity in linking. We test our algorithm in a challenging real worldsetting, where we automatically estimate scene context and intendedgoals, then track multiple people from a moving camera