Publication Bias and Evidential Value in Speech, Language, and Hearing Research

Purpose: This research examined the evidential value of research in Speech, Language, and Hearing (SLH), and the extent to which there is publication bias in reported findings. We also looked at the prevalence of good research practices, including those that work to minimize publication bias.Method: We extracted statistical results from 51 articles reported in four meta-analyses. These were there analyzed with two recent tests for evidential value and publication bias —the p-curve and the Z-curve. These articles were also coded for pre-registration, data access statements, and whether they were replication studies. Results: P-curves were right-skewed indicating evidential value, ruling out selective reporting as the sole reason for the significant findings. The Z-curve similarly found evidential value but detected a relative absence of null results, suggesting there is some publication bias. No studies were pre-registered, no studies had a data access statement, and no studies were full replication studies (3 studies were partial replications).Conclusions: Findings indicate SLH research has evidential value. This means that decision-makers and clinicians can continue to rely on the SLH research evidence base to influence service and clinical decisions. However, the presence of publication bias means that meta-analytic estimates of effectiveness may be exaggerated. Thus, we encourage SLH researchers to engage in study pre-registration, make result data accessible, conduct replication studies, and document null findings.