Public Disclosure of Results From Artificial Intelligence/Machine Learning Research in Health Care: Comprehensive Analysis of ClinicalTrials.gov, PubMed, and Scopus Data (2010-2023) (Preprint)

BACKGROUND Despite the rapid growth of research in artificial intelligence/machine learning (AI/ML), little is known about how often study results are disclosed years after study completion. OBJECTIVE We aimed to estimate the proportion of AI/ML research that reported results through ClinicalTrials.gov or peer-reviewed publications indexed in PubMed or Scopus. METHODS Using data from the Clinical Trials Transformation Initiative Aggregate Analysis of ClinicalTrials.gov, we identified studies initiated and completed between January 2010 and December 2023 that contained AI/ML-specific terms in the official title, brief summary, interventions, conditions, detailed descriptions, primary outcomes, or keywords. For 842 completed studies, we searched PubMed and Scopus for publications containing study identifiers and AI/ML-specific terms in relevant fields, such as the title, abstract, and keywords. We calculated disclosure rates within 3 years of study completion and median times to disclosure—from the “primary completion date” to the “results first posted date” on ClinicalTrials.gov or the earliest date of journal publication. RESULTS Of 842 completed studies (n=357 interventional; n=485 observational), 5.5% (46/842) disclosed results on ClinicalTrials.gov, 13.9% (117/842) in journal publications, and 17.7% (149/842) through either route within 3 years of completion. Higher disclosure rates were observed for trials: 10.4% (37/357) on ClinicalTrials.gov, 19.3% (69/357) in journal publications, and 26.1% (93/357) through either route. Randomized controlled trials had even higher disclosure rates: 11.3% (23/203) on ClinicalTrials.gov, 24.6% (50/203) in journal publications, and 32% (65/203) through either route. Nevertheless, most study findings (82.3%; 693/842) remained undisclosed 3 years after study completion. Trials using randomization (vs nonrandomized) or masking (vs open label) had higher disclosure rates and shorter times to disclosure. Most trials (85%; 305/357) had sample sizes of ≤1000, yet larger trials (n>1000) had higher publication rates (30.8%; 16/52) than smaller trials (n≤1000) (17.4%; 53/305). Hospitals (12.4%; 42/340), academia (15.1%; 39/259), and industry (13.7%; 20/146) published the most. High-income countries accounted for 82.4% (89/108) of all published studies. Of studies with disclosed results, the median times to report through ClinicalTrials.gov and in journal publications were 505 days (IQR 399-676) and 407 days (IQR 257-674), respectively. Open-label trials were common (60%; 214/357). Single-center designs were prevalent in both trials (83.3%; 290/348) and observational studies (82.3%; 377/458). CONCLUSIONS For over 80% of AI/ML studies completed during 2010-2023, study findings remained undisclosed even 3 years after study completion, raising questions about the representativeness of publicly available evidence. While methodological rigor was generally associated with higher publication rates, the predominance of single-center designs and high-income countries may limit the generalizability of the results currently accessible.