Combining eye and mouse movement measures to infer the accumulation of evidence across eye fixations in visual search with visually degraded scenes

International audienceVisual search and identification of an object in the environment can be seen as decision-making processes where sensory information is accumulated across eye fixations and where the expectations about the target object and its context are integrated. In traditional perceptual decision-making models, the accumulation of information is usually modelled as a passive process.This study focuses on the contribution of eye movements for the integration of expectations about the target (its identity, its location) with degraded sensory information (with fog or artificial noise) towards the accumulation of evidence. We used the mouse-tracking paradigm, allowing to infer dynamic aspects of the decision-making process through a computer mouse movements. Target detection (target latency) and verification were measured through eye movements during visual search tasks in visually degraded scenes. Mediation models were used to assess the respective contributions of target detection and verification in the decision-making process.In Experiment 1, we manipulated the expectations about target location during visual search in indoor visual scenes. The results suggest that evidence for the target absent response may be accumulated as the target is actively but unfruitfully searched for through eye movement. However, no clear conclusion can be drawn. Experiment 2 featured the visual search of a pedestrian in foggy driving-themed scenes, where we manipulated expectations about the presence of a pedestrian (target) or a roe deer (distractor). This allowed to infer the accumulation of evidence for the target absent response when a pedestrian (incorrect response) or a roe deer (correct response) was in the scene. Taken together, these studies emphasize the distinct contributions of target detection and verification in the accumulation of evidence toward the target present and target absent responses. Furthermore, they suggest an evidence accumulation-based mechanism addressing the search termination problem