Meta-analyzing non-preregistered and preregistered studies

Preregistration is gaining ground in psychology, and a consequence of this is that preregistered studies are more often included in meta-analyses. Preregistered studies likely mitigate the effect of publication bias in a meta-analysis, because preregistered studies can be located in the registries they were registered in even if they do not get published. However, current meta-analysis methods do not take into account that preregistered studies are less susceptible to publication bias. Traditional methods treat all studies as equivalent while meta-analytic conclusions can be improved by taking advantage of preregistered studies. The goal of this paper is to introduce the Hybrid Extended Meta-Analysis (HYEMA) method that takes into account whether a study is preregistered or not to correct for publication bias in only the non-preregistered studies. The proposed method is applied to two meta-analyses on prominent effects in the psychological literature: the red-romance hypothesis and money priming. Applying HYEMA to these meta-analyses shows that the average effect size estimate is substantially closer to zero than the estimate of the random-effects meta-analysis model. Two simulation studies tailored to the two applications are also presented to illustrate the method's superior performance compared to the random-effects meta-analysis model and PET-PEESE when publication bias is present. Hence, I recommend to apply HYEMA as a sensitivity analysis if a mix of both preregistered and non-preregistered studies are present in a meta-analysis. R code as well as a web application (https://rcmvanaert.shinyapps.io/HYEMA) have been developed and are described in the paper to facilitate application of the method.