A goal-driven deep learning approach for V1 system identification

Understanding sensory processing in the visual system results from accurate predictions of its neural responses to any kind of stimulus. Although great effort has been devoted to the task, we still lack a full characterization of primary visual cortex (V1) computations and their role in higher cognitive functional tasks (e.g. object recognition) in response to naturalistic stimuli. While previous goal-driven deep learning models have provided unprecedented performance on visual ventral stream predictions and revealed hierarchical correspondence, no study has used the representations learned by those models to predict single cell spike counts in V1. We introduce a novel model (Fig. 1A) that leverages these learned representations to build a linearized model with Poisson noise. We separately use the representations of each convolutional layer of a near-state of the art convolutional neural network (CNN) trained on object recognition to fit a model that predicts V1 responses to naturalistic stimuli. When fitted to data collected from neurons across cortical layers in V1 from an awake, fixating monkey, we found that, as we expected, intermediate early layers in the CNN provided better performance on held out data (Fig. 1B). Additionally we show that, using the best predictive layers, our model significantly outperforms classical and current state-of-the-art methods on V1 identification (Fig. 1C). When exploring the properties of the best predictive layers in the CNN, we found striking similarities with known V1 computation. Our model is not only interpretable, but also interpolates between recent subunit-based hierarchical models and goal-driven deep learning models leading to results that argue in favor of shared representations in the brain