Shape features underlying the perception of liquids

Visually identifying and estimating properties of materials is critical for many tasks, from reaching for a steaming kettle to walking on a wet marble staircase. From a computational point of view, liquids represent a particularly interesting class of materials because of their highly mutable shapes and physical complexity. The ways liquids move and change shape is affected both by external forces (e.g. object interactions, gravity) and intrinsic properties (e.g. viscosity, density). Estimating viscosity therefore requires disentangling the relative contributions of external and internal factors. Previous studies have shown that despite the wide spectrum of possible liquid shapes, observers are surprisingly good at estimating viscosity across variations in the external conditions. In this study we test the extent to which perceived viscosity can be predicted using mid-level shape and motion features (e.g. 'spread', 'piling up', 'angularity'). We conducted experiments using simulated liquids. Observers either rated viscosity or twenty shape features (e.g., volume, folding, spiraling), separately in three different sets: (1) A liquid pouring on a plane simulated with 32 steps of viscosity. (2) Eight random variations of this scene with seven different viscosities, created by perturbing the flow of the liquids with different force fields, leading to substantial shape differences over time. (3) Eight different scenes of various kinds of interaction (e.g., stirring, rain, waterfall) at seven different viscosities. Results indicate that viscosity ratings are well predicted through a linear combination of small numbers of mid-level features, as rated by other participants. In the first stimulus set, all twenty predictors yields R 2 = 0.99 and with the six main predictors R 2 = 0.98. Since the liquids are computer simulated we know the true 3D shape for each frame. From these we derived shape measurements that correlate with each of the six main predictors, resulting in predictions of perceived viscosity with R 2 = 0.92. Applying the same model with the same feature weights to the ratings on the second stimulus set (variations of pouring liquids) yields R 2 = 0.80, but the model loses much of its predictive power for the diverse scene variations of the third stimulus set (R 2 = 0.15). Looking at the results of shape feature ratings for the eight different scenes it becomes clear that rather than relying on a fixed set of features, the visual system flexibly identifies and re-weights mid-level cues depending on the context to achieve high levels of constancy. Combining data across all experiments we have isolated four shape features (Angular, Spread, Piling up, Complexity) that generalize quite well across contexts (mean R 2 = 0.97) but this requires a model that can re-weight the features according to constraints derived from each particular context. How our visual system estimates these constraints on a scene-by-scene basis remains the important open question.Perceptual Representation of Illumination, Shape and Materia