Flexpad: Highly Flexible Bending Interactions for Projected Handheld Displays

Flexpad is an interactive system that combines a depth camera and a projector to transform sheets of plain paper or foam into flexible, highly deformable, and spatially aware handheld displays. We present a novel approach for track-ing deformed surfaces from depth images in real time. It captures deformations in high detail, is very robust to oc-clusions created by the user’s hands and fingers, and does not require any kind of markers or visible texture. As a result, the display is considerably more deformable than in previous work on flexible handheld displays, enabling nov-el applications that leverage the high expressiveness of detailed deformation. We illustrate these unique capabilities through three application examples: curved cross-cuts in volumetric images, deforming virtual paper characters, and slicing through time in videos. Results from two user stud-ies show that our system is capable of detecting complex deformations and that users are able to perform them quick-ly and precisely. Author Keywords Flexible display; handheld display; tracking; projection