External validation of automated deep-learning based echocardiogram analysis

Abstract Background The use of artificial intelligence (AI) in echocardiography is constantly expanding. Several AI algorithms now enable fully automated measurement of echocardiographic parameters, which can potentially reduce measurement variability and increase efficiency. These AI algorithms have demonstrated accuracy comparable to conventional human-based measurements. However, to adopt automated AI echocardiography analysis for routine use, we need a deeper understanding of its performance compared with that of human expert readers. Purpose This study aims to perform a real-world external validation of a commercial, fully automated AI algorithm by comparing its performance standard to echocardiographic measurements done by human experts. Methods We retrospectively analyzed 889 consecutive hospitalized patients who underwent a clinically indicated transthoracic echocardiographic study. Each study was performed by experienced cardiac sonographers and reviewed by expert echo cardiologists. All studies were also analyzed by a commercial, fully automated AI software (Us2.AI). We analyzed the correlation between standard echocardiographic parameters measured by human experts to those measured by the AI algorithm. Results Of the 889 patients, 731 (82%) had sufficient echocardiographic data to be included in the analysis (mean age 68±16, 46% Female). As shown in Table 1, the measures with the strongest correlation were E wave velocity (r=0.96, P<0.001), E/e' ratio (r=0.86, P<0.001), and systolic pulmonary artery pressure (r=0.88, P<0.001) whereas posterior wall thickness and left ventricular ejection fraction (LVEF) had lower correlation coefficients (r=0.52 and r=0.61, respectively, both P<0.001). On Bland-Altman plot (Figure 1), mean AI-obtained LVEF values were significantly higher compared to human-estimated LVEF values with a mean difference of 5.8. This discrepancy was specifically evident in cases where the human-based LVEF was 45% or higher. Conclusion In this external validation study, AI-based echocardiographic measurements showed a strong correlation with human-based measurements. LVEF values obtained by AI tended to be higher than human LVEF estimation. Correlation of echocardiographic measure Bland Altman for LV EF