Use the force: how force touch improves input on handheld touchscreens

Handheld devices, such as smartphones, have become essential tools in our everyday life. We use them, e.g., to contact people, browse the web, or take pictures. For whatever use, to interact with the handheld device, we hold it with one or two hands and touch with our fingers on the built-in touchscreen. However, this interaction is often constrained to simple contact between the finger and the flat display glass, although touch offers further, richer properties. One such rich property is the intensity of a touch, i.e., its force, that the user applies with every tap to the touchscreen. Incorporating this property into the user’s interaction with the handheld device enables her to become more expressive with every single touch. In this thesis, we present a series of interaction techniques that take advantage of force touch input to make handheld interaction more efficient: When holding the device with two hands in landscape orientation, most of the fingers are unavailable for interaction, since they rest at the back of the device (BoD), holding it in place. Using BoD force input, we can make efficient use of these fingers without sacrificing stability of the device grip. Our technique, BackXPress, enables quick access to shortcuts and menus to augment users’ touch interaction with the frontal screen. For single-handed device use, users can only use their thumb to interact with the frontal touchscreen but cannot reach everywhere without re-grasping the device. Our virtual thumb extension, ForceRay, lets the user cast a ray at unreachable targets and control a cursor on that ray that moves closer to these targets the more force is applied. The technique is ergonomic for the thumb and enables users to maintain a steady device grip. Targets located at the screen edges, like menus and navigation buttons, are acquired quickly. Selection of values from long ordered lists, such as picking a date or time, can also be sped up when using force input. With our Force Picker, users scroll through the value range at various speeds, with the speed being coupled to the force exerted by the thumb. Prior rolling of the thumb to the left or right sets the scrolling direction. Compared to touch-based pickers, our Force Picker not only makes selection faster, but also only consumes little screen space since the gesture footprint for force input is much smaller. While controlling force via fingers requires practice, we show that with training and algorithmic optimizations, users become quickly familiar with force input and gain the benefits of the added expressiveness for handheld interaction