Tracking known three-dimensional objects

A method of visually tracking a known three-dimensional object is described. Predicted object position and orientation extrapolated from previous tracking data are used to find known features in one or more pictures. The measured image positions of the features are used to adjust the estimates of object position, orientation, velocity, and angular velocity in three dimensions, Filtering over time is included as an integral part of the adjustment, so that the filtering both smooths as appropriate to the measurements and allows stereo depth information to be obtained from multiple cameras taking pictures of a moving object at different times. I II’iiODUCI’ION Previous work in visual tracking of moving objects has dealt mostly with two-dimensional scenes Cl, 2, 31, with labelled objects [41, or with restricted domains in which only partial spatial information is extracted I51. This paper describes a method of tracking a known solid object for which an accurate object model is available, determining its three-dimensional position and orientation rapidly as it moves, by using natural features on the object. Only the portion of the tracking problem concerning locking onto an object and tracking it when given initial approximate data is discussed here. The acquisition portion of the problem is currently being worked on and will be described in a later paper. Since the tracking proper portion discussed here has approximate information available from the acquisition data or from previous tracking data, it can quickly find the expected features in the pictures, and it can be optimized to use these features to produce high accuracy, good coasting through times of poor data, and optimum combining of information obtained at different times. (An earlier, similar method lacking many of the features described here was previously reported 161.) mode 1 The current method uses a general object consi sting of planar surf aces. The f eature