Outlier detection and rejection in scatterplots: Do outliers influence intuitive statistical judgments?

According to a growing body of research, human adults are remarkably accurate at extracting intuitive statistics from graphs, such as finding the best-fitting regression line through a scatterplot. Here, we ask whether humans can also perform outlier rejection, a non-trivial statistical problem. In three experiments, we investigated human adults’ capacity to evaluate the linear trend of a flashed scatterplot comprising 0-4 outlier datapoints. Experiment 1 showed that participants did not spontaneously reject outliers: when outliers were not mentioned, their presence biased the participants’ trend judgments and regression line estimates. In experiment 2, where participants were explicitly asked to exclude outliers, the outlier-induced bias was reduced but remained significant. In experiment 3, where participants were asked to explicitly detect any outlier before adjusting their regression line, outlier detection was satisfactory, but the detected outliers continued to bias the regression responses, unless they were quite distant from the main regression line. We propose a simple model for outlier detection, according to which humans detect outliers by computing a z-score that estimates how far a given datapoint is from the distribution of distances to the regression line. Detection is not rejection, however, and our results suggest that humans can remain biased by outliers that they have detected.