Abstract 1928: Prediction of neoadjuvant treatment outcomes with multimodal data integration in breast cancer

Abstract Neoadjuvant chemotherapy (NAC) is the standard of care for selected patients with high-risk early-stage breast cancer with pathologic complete response (pCR) being the most prominent predictor of favorable outcomes. Here, we sought to study the predictive capacity of integrating orthogonal diagnostic measures on predicting pCR relative to standard clinicopathologic features. We developed a computational model integrating radiology and pathology images, and tumor genomics to automatically predict pCR from multimodal data. We present an interim analysis on a cohort of 957 patients with at least one available pre-NAC data modality. The baseline AUC for pCR prediction by a trained and tested logistic regression model on 857 patients using standard clinicopathologic features including receptor subtype, demographic information, and stage was 0.77. MR images were input into a convolutional neural network (CNN) and a radiomics model. The trained CNN and radiomics models using selected images of 576 patients with pre-NAC MR images achieved AUCs of 0.65 and 0.60 on 164 hold-out test cases, respectively. We trained a multiple instance learning-based weakly supervised learning (MIL-WSL) model using 537,762 extracted tiles from whole slide images (WSI) of digital histopathology scans from 522 patients. The MIL-WSL model achieved AUC of 0.63 for pCR prediction on a hold-out test set of pre-NAC biopsies from 239 patients. A feature based classifier trained on 76 cases using tumor genomic features such as mutational burden, microsatellite instability, fraction genome altered, ploidy, purity, mutation and copy number alterations in selected genes achieved an AUC of 0.72 on 83 hold-out test cases. We then combined unimodal radiology, histopathology, and genomic predictions in a deterministic manner. This multimodal combination on an independent 68-patient test set achieved an AUC of 0.84, indicating increased power to resolve pCR than any modality alone, and over clinicopathologic baseline. Together, we present approaches to train models end-to-end using tensor fusion networks and attention-gating combined with MIL. Automated multimodal methods are here shown to improve prediction over established clinical parameters alone, motivating our ongoing efforts to refine and improve the model so as to achieve higher levels of efficiency. We anticipate these interim results will be further improved through refinement of input features and increasing the number of patients included in the final validation cohort. Citation Format: Pegah Khosravi, Elizabeth J. Sutton, Justin Jee, Timothy Dalfonso, Christopher J. Fong, Doori Rose, Edaise M. Da Silva, Armaan Kohli, David Joon Ho, Mehnaj S. Ahmed, Danny Martinez, Anika Begum, Elizabeth Zakszewski, Andrew Aukerman, Yanis Tazi, Katja Pinker-Domenig, Sarah Eskreis-Winkler, Atif J. Khan, Edi Brogi, Elizabeth Morris, Sarat Chandarlapaty, George Plitas, Simon Powell, Monica Morrow, Larry Norton, Jianjiong Gao, Mark Robson, Hong Zhang, Sohrab Shah, Pedram Razavi, MSK-MIND Consortium. Prediction of neoadjuvant treatment outcomes with multimodal data integration in breast cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1928.