External Validity

A threat to external validity is any factor that limits the generalizability of an observed result. Unlike all threats to statistical conclusion and internal validities and some threats to construct validity, threats to external validity cannot ordinarily be controlled by design. Nor is there any disagreement on how threats to external validity should be controlled. In most instances, it can only be controlled by replication?—across subjects, situations and time frames. This seldom happens, unfortunately, because the academic incentive structure discourages replication. The contemporary “reproducibility crisis” was spurred by a collaborative group of social scientists attempting to replicate one hundred experimental and correlational studies published in three mainstream psychology journals. Sixty percent of replications failed to reproduce the published effect. Failures to control for threats to external validity that stem from uncontrolled variations in persons, situations, and time frames, parsimosniously explain the failure rate in this replication study.