Test-statistic correlation and data-row correlation

Abstract When a statistical test is repeatedly applied to rows of a data matrix—such as in differential-expression analysis of gene expression data, correlations among data rows will give rise to correlations among corresponding test statistic values. Correlations among test statistic values create many inferential challenges in false-discovery-rate control procedures, gene-set enrichment analysis, or other procedures aiming to summarize the collection of test results. To tackle these challenges, researchers sometimes will—explicitly or implicitly—use the correlations (e.g., as measured by the Pearson correlation coefficients) among the data rows to approximate the correlations among the corresponding test statistic values. We show that, however, such approximations are only valid under limited settings. We investigate the relationship between the correlation coefficient between a pair of test statistics (test-statistic correlation) and the correlation coefficient between the two corresponding data rows (data-row correlation). We derive an analytical formula for the test-statistic correlation as a function of the data-row correlation for a general class of test statistics: in particular, two-sample t -test is a special case. The analytical formula implies that the test-statistic correlation is generally weaker than the corresponding data-row correlation, and in general, the latter will not well approximate the former when the involved null hypotheses are false. We verify our analytical results through simulations.