Visual Observation of a Moving Agent

We address the problem of observing a moving agent. In particular, we propose a system for observ-ing a manipulation process, where a robot hand maliipulates an object. A discrete event dynamic system (DEDS) frame work is developed for the handlobject interaction over time and a stabilizing observer is constructed. Low-level modules are developed for recognizing the "events " that causes state transitions within the dynamic manipulation system. The work examines closely the pos-sibilities for errors, mistakes and uncertainties in the manipulation system, observer construction process and event identification mechanisms. The system utilizes different tracking techniques in order t o observe and recognize the task in an active, adaptive and goal-directed manner.