Abstract 393: Generalizability of Deep Learning Models for Aneurysm Detection: A Systematic Review of Cross‐Center and Cross‐Modality Validation Performance

Introduction Deep learning (DL) has demonstrated high diagnostic accuracy in detecting intracranial aneurysms (IAs) on CT angiography (CTA) and digital subtraction angiography (DSA). However, most studies rely on single‐center retrospective datasets, raising concerns about model generalizability across institutions and imaging modalities. This systematic review and meta‐analysis evaluates the external validity of DL models for IA detection, with emphasis on cross‐center and cross‐modality performance. Methods We systematically reviewed meta‐analyses, retrospective multicenter validations, and prospective trials that assessed DL algorithms for IA detection using internal versus external datasets. Extracted metrics included sensitivity, specificity, and area under the curve (AUC). Comparative analysis was performed between internal validation (training/derivation cohorts) and external validation (independent centers or modalities). Subgroup analysis focused on segmentation performance (Dice similarity coefficients, DSC) and prospective clinical deployment. Results Four representative studies encompassing over 15,000 patients were analyzed. Overall, DL models demonstrated excellent internal performance, with pooled sensitivity of 0.93 (95% CI: 0.90‐0.95) and AUC 0.94 (95% CI: 0.92‐0.96). However, external validation revealed modest performance drops. Wei et al. (2024) reported sensitivity of 0.88 internally versus 0.857 externally, with AUC decline from 0.95 to 0.93. You et al. (2024) achieved internal sensitivity of 0.962 and Dice coefficient of 0.78, but external validation yielded reduced sensitivity (0.94) and Dice 0.71. Prospective validation by Hu et al. (2024) confirmed robust generalizability, with sensitivity of 0.957 internally and 0.943 externally, and consistent AUC (0.909 across both). Din et al. (2023) highlighted systematic reductions in accuracy when models were applied to unseen centers, with pooled sensitivity decreasing from 0.91 (internal) to 0.86 (external), reflecting potential overfitting. Across studies, pooled sensitivity loss on external testing averaged ‐4.5%, while AUC declined by ‐0.03. Despite this, all models retained diagnostic accuracy comparable to radiologists, underscoring strong translational potential. Conclusion Deep learning models for IA detection demonstrate strong internal validity but experience modest reductions in sensitivity and AUC when tested across centers and modalities. Performance declines are most evident in segmentation accuracy, with Dice coefficients dropping by ∼0.07 on external validation. Nonetheless, prospective multicenter trials confirm clinical robustness, with external sensitivity >94% in some models. These findings highlight both the promise and the challenge of achieving true generalizability. Future research should prioritize harmonized multi‐institutional datasets, standardized validation frameworks, and robust cross‐modality testing to ensure reliable deployment in diverse clinical environments.