Mapping the landscape of multiverse-style data-analytic methods: A scoping review

Multiverse-style methods of data analysis try to account for all possible decisions that could be made by researchers during their analyses and show how those often arbitrary decisions lead to different results. A total of 424 sources were included in this first comprehensive scoping review on this topic, giving an overview of all applications and developments of multiverse-style data-analytic methods. The constant rise in applications of these methods in diverse scientific disciplines spurred developments in all their aspects. Inconsistencies could be found in terminology and, analysis and reporting of results of multiverse-style methods. Although preregistration still falls short, analysis code was available in almost half of all included sources. Concluding recommendations on good practice might further increase the standardisation and transparency of the methodology and provide a first guideline for multiverse-style methods.