3D Cinema Survey Virtualized Reality: Perspectives on 4D Digitization of Dynamic Events

Digitally recording dynamic events, such as sporting events, for experiencing in a spatiotemporally distant and arbitrary setting requires 4D capture: three dimensions for their geometry and appearance over the fourth dimension of time. Today’s computer vision techniques make 4D capture possible. The Virtualized Reality system serves as an example in this discussion on the general problem of digitizing dynamic events. Dynamic events spark great interest for entertainment, education, and exploration. In traditional media, each viewer observes the event from a fixed viewpoint determined by the producer or the viewer’s seating position. Can we digitally record a dynamic event and experience it in a spatio-temporally flexible manner? A viewer can watch a sporting event from any viewpoint—for example, a player’s or even the ball’s. An audience member can view a ballet from a virtual seat in the middle of the stage. The recording can enhance the training experience so that medical personnel can visit a particularly difficult step repeatedly from the most informative viewpoint. We can also edit or modify the recorded event in creative ways, such as changing its appearance after capture or creating a new event by combining parts of recorded events. Such a medium of capturing and manipulating digitized dynamic events can open new vistas in immersive and participative entertainment, empowering viewers to control their experience. Telepresence is the ability to be present at an event taking place at a distance. If we can digitize a dynamic event and let a user immerse into it, we can achieve tele-experience (if live) or post-experience (if delayed), navigating through and interacting with the digitized event. The ability to experience a remote dynamic event in its richness, unhindered, arguably can provide the functional equivalent of teleportation. In the early 1990s, we began developing multicamera computer-vision technologies at Carnegie Mellon to digitize large, dynamic events. The CMU system comprised a large number of cameras to capture an event inside a room from all directions. It produced the 4D even