Standalone commercial artificial intelligence software for pulmonary nodule detection on chest radiographs: a systematic review

Abstract Background Pulmonary nodules may be missed on chest radiographs despite their clinical relevance for early lung cancer detection. Commercial artificial intelligence (AI) systems are increasingly available for chest radiograph interpretation, but evidence for their standalone performance remains fragmented across products, datasets, operating thresholds, and comparator designs. Purpose To systematically evaluate the standalone diagnostic performance of commercially available AI software for pulmonary nodule detection on chest radiographs compared with radiologists reading alone, and to identify methodological factors shaping interpretation of the evidence. Methods A systematic review was conducted according to PRISMA 2020 guidelines and registered with PROSPERO (CRD420261383905). PubMed, PubMed Central, Scopus, and Web of Science were searched. Eligible studies evaluated commercially available AI systems as standalone detectors for pulmonary nodules on chest radiographs and directly compared their performance with radiologists reading alone using an accepted clinical reference standard. Risk of bias was assessed using an adapted QUADAS-2 framework. Findings were synthesized narratively because of heterogeneity in study design, populations, reference standards, operating thresholds, and outcome measures. Results Six studies met the inclusion criteria, comprising 12 product-level evaluations across 10 commercial AI systems. Standalone AI performance varied: some evaluations showed AI exceeding radiologist-level performance, others showed no significant difference, and one showed AI below individual reader performance with approximately twice the false-positive rate. In the only standardized within-study comparison of multiple commercial products on the same dataset, four of seven systems significantly exceeded mean radiologist AUC. Standalone AI AUC ranged from 0.79 to 0.93 against a mean reader AUC of 0.81, demonstrating that performance varied even among cleared products tested under identical conditions. Interpretation was limited by differences in operating thresholds, comparator design, reference standards, study populations, and outcome reporting. Risk-of-bias concerns were common, particularly in patient selection. Conclusions In the available peer-reviewed evidence, standalone commercial AI systems for chest radiograph pulmonary nodule detection show variable performance. Evidence remains insufficient to support class-wide standalone deployment because all studies were retrospective or reader-study based, thresholds were inconsistently reported, and real-world false-positive burden remains uncertain. Prospective real world evaluation with clearly reported operating thresholds, standardized comparator designs and complete diagnostic performance are needed.