The Systematic Multiverse Analysis Registration Tool (SMART) for Defining Multiverse Analyses

Multiverse analysis is increasingly recognised as a systematic framework for investigating and reporting the robustness of scientific results across alternative defensible data processing and analysis pipelines. Defining the multiverse of alternative pipelines requires identifying all defensible combinations of options across multiple decision nodes along the data analysis workflow. This remains a cognitively demanding and logically complex task. Consequently, the extent of transparent documentation of the decisions made to design the multiverse analysis and the rationale for those decisions is variable and not always satisfactory in the current literature. A lack of transparent and systematic documentation of this process risks undermining the core methodological goal of communicating uncertainty. While preregistration of experimental studies is increasingly popular and is recognised for increasing transparency and trust in the research process, these well-known benefits are not yet fully harnessed for multiverse analyses. To address this, we developed the Systematic Multiverse Analysis Registration Tool (SMART), designed to support researchers from different scientific disciplines to construct multiverse analyses in a transparent and systematic stepwise process. SMART guides users through each step of the multiverse construction procedure, documenting all decisions and justifications at each step. It supports the construction of multiverse analyses that include all defensible pipelines within a given scope, and principled multiverse analyses where the defensible multiverse is deflated based on objective criteria of pipeline equivalence. The documentation is exportable and can be used as a preregistration or as supplementary material to the published multiverse analysis. By increasing the ease, transparency and reproducibility of multiverse construction, SMART facilitates more rigorous robustness assessments and contributes to the broader goals of open and reproducible science.