Toward reproducible evaluation of reproducibility for biomedical modelling

Presentation delivered at the UC Meta Colloquium, November 16-18 2022 Abstract: The Center for Reproducible Biomedical Modeling (https://reproduciblebiomodels.org/) aims to enable comprehensive predictive models of biological systems that can guide precision medicine and synthetic biology. The centre is making models understandable, reproducible, reusable, and composable by (1) developing missing model building, annotation, and simulation tools and (2) combining these and other existing tools into a user-friendly reproducible modelling workflow. Ultimately, we believe this workflow will help modelers create comprehensive models that can guide medicine and bioengineering. Here I will focus primarily on our attempts to formalise the evaluation of reproducibility of biomedical models in the literature. I will discuss a pilot that we ran with the PLoS Computational Biology journal where we performed our reproducibility evaluation as part of the traditional peer review workflow (https://doi.org/10.1371/journal.pcbi.1007881). Preliminary analysis of this pilot has presented some interesting observations leading to speculation regarding entrenched practices in scientific publication and academic career progression, and I’ll mention various initiatives trying to address these.