Multiverse analyses in the classroom

Most empirical papers in psychology involve statistical analyses performed on a new or existing dataset. Sometimes the robustness of a finding is demonstrated via data-analytical triangulation (e.g., obtaining comparable outcomes across different operationalizations of the dependent variable), but systematically considering the plethora of alternative analysis pathways is a rarity. In contrast, researchers conducting a multiverse analysis precisely aim to assess the robustness of a finding by methodically examining the seemingly-arbitrary choices pertaining to data processing and/or model building. In the present paper, we describe how the multiverse approach can be successfully implemented in student research projects within psychology programs, drawing on personal experience as instructors. Embedding a multiverse project in students’ curricula addresses an important scientific need, as studies examining the robustness or fragility of phenomena are largely lacking in psychology. More importantly, it offers students an ideal opportunity to put various statistical methods into practice, thereby also raising awareness about the abundance and consequences of arbitrary decisions in data-analytic processing. An attractive practical feature is that one can reuse existing datasets, which proves especially useful when resources are limited, or when circumstances such as the COVID-19 lockdown measures restrict data collection possibilities.