Assessment of visual, manual, and automatic coronary calcium scoring methods - analysis of dedicated CT scans and low dose CT scans acquired during cardiac 13N-ammonia-PET/CT

Abstract Funding Acknowledgements Type of funding sources: None. Background Coronary artery calcium (CAC) is a well-known predictor of major adverse cardiac events and is scored manually from dedicated, ECG-triggered CT scans. In the present study, we investigated the accuracy of risk categorisation based on visual and automatic AI calcium scoring from low dose CT (LDCT) scans and dedicated Calcium Score CT (CSCT) scans. Purpose To assess the agreement of risk prediction based on visual and automatic AI CAC scoring from CSCT scans and LDCT scans as compared to a gold standard, manual CSCT scoring. Methods We retrospectively enrolled 222 patients. Each patient received a 13N-ammonia PET with LDCT and CSCT scan. The time interval between LDCT and CSCT was less than 6 months. Each LDCT and CSCT scan was scored visually, manually, and automatically with AI. For visual scoring we used a previously described 6–points scale (0; 1-10; 11-100; 101-400; 401-100; >1000 Agatston score). For manual scoring we used a generally available software package (Syngo.via,Siemens). The automatic AI scoring was performed with commercially available software based on a deep learning algorithm (included in Syngo.via,Siemens). Each manually and automatically measured Agatston score was converted into the 6-points scale. We performed a per patient analysis; the risk group categorization was based on the total Agatston score. Spearman correlation coefficient was used to analyse the association between manual and automatic AI scoring methods. Agreement between visual, manual, and automatic AI scoring methods was determined using weighted kappa test with 95% confidence intervals (95%CI). Results The correlation between manual scoring from LDCT and CSCT scans was 0.96 (p < 0.001).The agreement between manual scoring from two scans, however, was low with weighted kappa equal 0.57 (95% CI 0.51 – 0.63). 91,9% of calcium scores measured by AI software on CSCT were in the same risk group as manual CSCT scores.The agreement between AI scoring and manual scoring using CSCT was excellent, the weighted kappa was equal 0.95 (95% CI 0.92 - 0.97).Based on visual scoring on LDCT scans, 74,3% of the scores were in the same category as manual scoring on CSCT scans. The agreement between the visual scoring on LDCT scans and a gold standard was strong, weighted kappa equal was 0.82 (95% CI 0.77 – 0.86). The agreement between manual and automatic scoring on LDCT using manual CSCT as the gold standard was low (0.57, 95 % CI 0.51 – 0.63; 0.49, 95 % CI 0.43 – 0.56, respectively). Based on visual LDCT scoring, 7 patients were incorrectly classified as calcium score 0, which underestimated the overall patients’ risk.The AI method scoring CSCT scans, classified 2 patients incorrectly as non-calcium risk group. Conclusions CAC can be automatically assessed from CSCT scans with commercially available AI software.Of manual, automatic, and visual CAC scoring on LDCT scans the visual scoring showed the highest agreement with the gold standard manual CSCT CAC scoring.