From p-values to Bayes Factor: A Meta-Analytic Comparison in Colorectal Research

Abstract The prevalent method for synthesizing evidence from multiple studies is the frequentist meta-analysis, which relies on assumptions of long-term frequencies and does not directly address the probability of hypotheses. In contrast, the Bayesian meta-analysis provides a framework that integrates prior knowledge with observed data, offering a more nuanced interpretation. This study aims to compare the outputs and interpretations of frequentist and Bayesian meta-analyses using published trials on colorectal anastomosis as examples. Two previously published meta-analyses on colorectal anastomosis—one evaluating trans-anastomotic tubes (TAT) and the other indocyanine green (ICG) fluorescence imaging—were reanalysed using frequentist and Bayesian approaches. Sequential Bayesian analyses were also conducted, updating priors with each additional study. Results were presented using odds ratios (OR), confidence intervals (CI), credible intervals (CrI), p-values, and Bayes factors (BF10). Both methods produced nearly similar ORs for the TAT meta-analysis; however, the Bayesian approach yielded slightly narrower CrIs and a BF10 that indicated a slight preference for the null hypothesis that was unclear with p-values alone. In the ICG meta-analysis, the Bayesian analysis produced a BF10 suggesting that it was 19 times more likely to observe the data under the assumption that the alternative hypothesis is true compared to the null, considerably making the estimates more conservative than the frequentist output. The Bayesian sequential analysis demonstrated increasing confidence in the alternate hypothesis with the addition of more studies. While frequentist and Bayesian meta-analyses may produce similar point estimates based on prior evidence, their interpretations and implications for hypothesis testing differ significantly. Bayesian methods offer a more flexible and intuitive approach, particularly in contexts with prior knowledge or when sequential updating is required. While frequentist outputs depend on multiple experiments, assuming that the null is true, and heavy dependence on conventional p-value thresholds, Bayesian outputs provide the direct probability of the hypothesis in question and credible intervals that are likely to contain the true estimate.