A long journey into reproducible computational neuroscience research

International audienceIn a previous modelling study, Leblois et al. (2006) demonstrated an action selection mechanism in cortico- basal ganglia loops based on competition between the positive feedback, direct pathway through the striatum and the negative feedback, hyperdirect pathway through the subthalamic nucleus. In Guthrie et al. (2013), authors investigated how multiple level action selection could be performed by the basal ganglia. To do this, the model has been extended in a manner consistent with known anatomy and electro-physiology in three main areas. Unfortunately, the information provided by the article were not sufficient to reproduce the model. If reproducibility is the hallmark of Science, non-reproducibility seems to be the hallmark of Computational Neurosciences. In that respect, Guthrie et al. (2013) is a prototypic case of such non-reproducible computational neuroscience research even though the proposed model gives a fair account of decision making in the basal ganglia complex. While trying to replicate results starting from the article description, we soon realised some information were undisclosed, some other were ambiguous and there were also some factual errors. Even after accessing the original sources (more than 6000 lines of Pascal), we were still unable to understand how the model worked. In the end, only the original material (a binary executable) and a direct contact with the authors allowed us to access the whole picture. After two months of intensive refactoring, we were finally able to replicate results using only 200 lines of Python. From this experience, which is unfortunately not an isolated case, we would like to share a simple message with the computational neuroscience community: designing computational models is not all about writing & running programs. If a model is to be reviewed, understood, used, replicated and integrated, it requires a minimal amount of coordinated efforts. Or the model will be soon forgotten, even by their own original designers