Exploring connections between similarity and categorization using vision and touch

Similarity has been proposed as a fundamental principle underlying the formation of category structures, however, much of the research involving perceptual similarity and categorization has focused on a single modality, usually vision. Can the notion of similarity still provide a basis for explaining categorization when objects are perceived with one or more modalities? We addressed these questions by having subjects see, touch, or both see and touch novel, 3D objects which varied parametrically in shape and texture. Then, they performed a pair-wise similarity rating task and a free sorting categorization task. Using multidimensional scaling (MDS), we found that that a single underlying perceptual map whose dimensions corresponded to shape and texture could explain visual, haptic, and bimodal similarity ratings. However, the relative weights of the map’s dimensions varied according to modality: shape dominated texture when objects were seen, whereas shape and texture were equally important in the haptic and bimodal conditions. We found some evidence for a connection between similarity and categorization in a multimodal context: the probability of category membership increased with similarity, while the probability of a category boundary being placed between two stimuli decreased with similarity. Moreover, the relative weight accorded to shape and texture varied in the same way for both tasks when modality was changed. The study also demonstrates how 3D printing technology and MDS techniques can be fruitfully applied in the study of visuohaptic object processing