BioModels Reproducibility Scorecard

Systems biology modelling involves the mathematical representation of biological processes to study complex system behaviour and was expected to be least affected by the reproducibility crisis. However, models often fail to reproduce and the reasons for the failure and prevalence were not fully understood. In a recent study, we analyzed 455 kinetic models published in 152 peer-reviewed journals. Most of these models were manually encoded from scratch to assess the reproducibility. Our investigation revealed that 49% of the models could not be reproduced using the information provided in the manuscripts. Among the corresponding authors, we contacted over 70% did not respond. Models across several life science journals failed to reproduce, revealing a common problem in the peer-review process. As a solution, we proposed a simple easy-to-use reproducibility scorecard with 8 yes-or-no questions that can be used by model authors, reviewers and journal editors during the peer-review process. We demonstrated that the scorecard is a good indicator of model reproducibility. In this workshop we discuss about our study in details, run a hands-on training to use the scorecard to assess models during the peer-review process. The reproducibility crisis in systems biology modelling can be tackled as a community, where model authors, reviewers, journal editors and funding bodies embrace reproducibility more proactively than before.