The NERVE-ML (neural engineering reproducibility and validity essentials for machine learning) checklist: ensuring machine learning advances neural engineering*

Abstract Objective. Machine learning’s (MLs) ability to capture intricate patterns makes it vital in neural engineering research. With its increasing use, ensuring the validity and reproducibility of ML methods is critical. Unfortunately, this has not always been the case in practice, as there have been recent retractions across various scientific fields due to the misuse of ML methods and validation procedures. To address these concerns, we propose the first version of the neural engineering reproducibility and validity essentials for ML (NERVE-ML) checklist, a framework designed to promote the transparent, reproducible, and valid application of ML in neural engineering. Approach. We highlight some of the unique challenges of model validation in neural engineering, including the difficulties from limited subject numbers, repeated or non-independent samples, and high subject heterogeneity. Through detailed case studies, we demonstrate how different validation approaches can lead to divergent scientific conclusions, highlighting the importance of selecting appropriate procedures guided by the NERVE-ML checklist. Effectively addressing these challenges and properly scoping scientific conclusions will ensure that ML contributes to, rather than hinders, progress in neural engineering. Main results. Our case studies demonstrate that improper validation approaches can result in flawed studies or overclaimed scientific conclusions, complicating the scientific discourse. The NERVE-ML checklist effectively addresses these concerns by providing guidelines to ensure that ML approaches in neural engineering are reproducible and lead to valid scientific conclusions. Significance. By effectively addressing these challenges and properly scoping scientific conclusions guided by the NERVE-ML checklist, we aim to help pave the way for a future where ML reliably enhances the quality and impact of neural engineering research.