Deep Learning and Minimally Invasive Endoscopy: Panendoscopic Detection of Pleomorphic Lesions

Introduction: Capsule endoscopy (CE) is a minimally invasive exam suitable of panendoscopic evaluation of the gastrointestinal (GI) tract. Nevertheless, CE is time-consuming with suboptimal diagnostic yield in the upper GI tract. Convolutional neural networks (CNN) are human brain architecture-based models suitable for image analysis. However, there is no study about their role in capsule panendoscopy. Methods: Our group developed an artificial intelligence (AI) model for panendoscopic automatic detection of pleomorphic lesions (namely vascular lesions, protuberant lesions, hematic residues, ulcers, and erosions). 355,110 images (6,977 esophageal, 12,918 gastric, 258,443 small bowel, 76,772 colonic) from eight different CE and colon CE (CCE) devices were divided into a training and validation dataset in a patient split design. The model classification was compared to three CE experts’ classification. The model’s performance was evaluated by its sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and area under the precision-recall curve. Results: The binary esophagus CNN had a diagnostic accuracy for pleomorphic lesions of 83.6%. The binary gastric CNN identified pleomorphic lesions with a 96.6% accuracy. The undenary small bowel CNN distinguished pleomorphic lesions with different hemorrhagic potentials with 97.6% accuracy. The trinary colonic CNN (detection and differentiation of normal mucosa, pleomorphic lesions, and hematic residues) had 94.9% global accuracy. Discussion/Conclusion: We developed the first AI model for panendoscopic automatic detection of pleomorphic lesions in both CE and CCE from multiple brands, solving a critical interoperability technological challenge. Deep learning-based tools may change the landscape of minimally invasive capsule panendoscopy.