A Bio-Inspired Evaluation Methodology for Motion Estimation

International audienceEvaluation of neural computational models of motion perception currently lacks a proper methodology for benchmarking. Here, we propose an evaluation methodology for motion estimation which is based on human visual performance, as measured in psychophysics and neurobiology. Offering proper evaluation methodology is essential to continue progress in modeling. This general idea has been very well understood and applied in computer vision where challenging benchmarks are now available, allowing models to be compared and further improved. The proposed standardized tools allow to compare different approaches, and to challenge current models of motion processing in order to define current failures in our comprehension of visual cortical function. We built a database of image sequences to depict input test cases corresponding to displays used in psychophysical settings or in physiological experiments. The data sets are fully annotated in terms of image and stimulus size and ground truth data concerning dynamics, direction, speed, etc. Since different kinds of models have different kinds of representation and granularity, we had to define generic outputs for each considered experiment as well as correctness evaluation tools. We propose to use output data generated by the considered model as read out that relates to observer task or functional behavior. Amplitude of pursuit or direction likelihoods are two examples. We probed several models of motion perception by utilizing the proposed benchmark The employed models show very different properties and internal mechanisms, such as feedforward normalizing models of V1 and MT processing and recurrent feedback models. Our results demonstrate the usefulness of the approach by highlighting current properties and failures in processing. So we provide here a valuable tool to unravel the fundamental mechanisms of the visual cortex in motion perception. The complete database as well as detailed scoring instructions and results derived by investigating several models are available at http://www-sop.inria.fr/neuromathcomp/software/motionpsychobenc