The live paper system

For the past 30 years, most interactions with computers have been through monitors, keyboards, and more recently mice. However, with ever increasing and more ubiquitous computing power, it is important to research other interaction modalities that are simultaneously richer and easier to use. By using computers to augment objects with illusionary intelligence and capabilities, it is possible for people to use these objects in new and exciting ways. Video cameras allow computers to monitor an environment non-intrusively, and react when appropriate. -- This thesis presents a realized Video Augmented Environment, called Live Paper, for enhancing paper documents on a tabletop. To the user, it appears as if the paper gains new visual and auditory features. A piece of paper can be removed and returned to the tabletop, and it will regain the same features it previously exhibited. A data projector, connected to the computer, projects annotations onto the paper and the tabletop. The computer uses a video camera, mounted next to the projector, to capture images. The system analyzes the images to determine page locations, orientations, and identifications. Thus the projected annotations appear locked to the page, moving and rotating as the user moves and rotates the page. -- Live Paper provides several page-based applications, called transparencies, including a 3D architecture visualizer, a music player, and a collaboration tool. All transparencies use a consistent set of projected buttons, with which the user interacts using his or her finger. -- The page-finding algorithms extract features from the captured images by segmenting (using Otsu grey-level thresholding) and analyzing the boundaries (using Freeman-Davis comer finding). The feature analysis stage examines these boundaries to find the pages. The thesis also presents the successful analysis of overlapping and occluded pages, based on a new application of perceptual occlusion techniques. -- To identify paper, Live Paper uses a novel method that does not require fiducials, works in non-ideal conditions, and uses low resolution video images. The system determines the similarity of found and stored pages using a modified Hausdorff Distance. The thesis presents a comparison of the Hausdorff measure with a Euclidean measure, using a set of sample pages including hand-written pages, presentations, journals, and paintings. -- The thesis reviews the transformations used to synchronize the locations from the captured image with the internal models and the projected workspace. The system design section includes details on how the hardware and software modules interact, and how programmers can add additional transparencies to the system. -- The Live Paper system successfully demonstrates a unique augmented environment where ordinary paper is the interface with the computer. This thesis lays a foundation for future research and development of a general purpose augmented tabletop