Sensing the Senses: Multimodal Research with Applications in Product Design

When people interact with products, all their sensory modalities are open to receive information. However, not all sensory information necessarily communicates the same message. To understand product experience, designers must know these incoming sensory messages, whether perceived consciously or not. Therefore, we studied (1) the roles individual modalities play in product experiences (2) the correspondences between modalities and (3) the ways people integrate sensory information. In a split-modality experiment, subjects perceived products through a single modality. Vision and touch provided the most detailed product information, followed by audition and, with a distance, olfaction. Also, vision and touch yielded the clearest expectations of what other modalities were likely to perceive. In a study of cross-modal correspondences, subjects matched smells to colors consistently, resulting in characteristic color profiles for different fragrances. These smell-color correspondences seem to be partly mediated by the emotions people experience. In studies of multimodal integration, the relative importance of modalities depended on the product and the type of assessment subjects made. We are now developing a practical design approach that optimizes multimodal unity in a product. First, we choose the experience the product should evoke. Second, consumer research determines the sensory impressions associated with this experience. Third, the designer integrates the selected sensory elements with the technical specifications into a product. Insight into the relative importance of the modalities assists in establishing design priorities