Prakash: Lighting Aware Motion Capture using Photosensing Markers and Multiplexed Illuminators

Figure 1: Motion and incident illumination capture in a natural setting with imperceptible marker tags. (Left) Traces of hidden tags superimposed on an actor running in sunlight. (Middle) A video frame from a recording session. (Right) A virtual sword inserted with matching location, orientation, lighting and motion blur. In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets