Abstract P1-03-16: Multimodal analyses of clinical, radiology, pathology and genomic information for enhanced prediction of response to neoadjuvant therapy in breast cancer

Abstract Background: Neoadjuvant chemotherapy (NACT) is the standard of care for early-stage breast cancer patients (pts) with high-risk clinical features and pathologic complete response (pCR) is considered the best predictor of favorable long-term outcomes. We developed machine learning models to predict pCR from a series data modalities available at the time of diagnosis. We compared the predictive value of unimodal radiology, pathology, genomic, and clinical features, and their multimodal integration to improve pCR prediction. Methods: We analyzed a multimodal cohort of 1,192 early-stage breast cancer pts treated with NACT at Memorial Sloan Kettering (MSK) between 2014 and 2021. Clinicopathologic features including hormone receptor subtype, demographic information (self-reported race, age at diagnosis), grade and stage were collected and curated for all pts, and combined to build a logistic regression prediction model. Digital pathology whole-slide images (WSIs) from pre-NACT tumor biopsies were collected for 1,101 patients. We used a pre-trained transformer model and attention-based multiple instance learning (MIL) to predict pCR from H&E WSIs. Pre-NACT magnetic resonance imaging (MRI) exams, which included T2-weighted images, T1-weighted precontrast images and T1-weighted post-contrast images, were collected for 838 pts, and were used to build a deep learning (DL) model to predict pCR. Tumor biopsies and matched blood specimens from 315 pts underwent targeted DNA sequencing using the MSK-IMPACT platform, which identified mutations, copy number changes, and structural rearrangements in a selected panel of up to 505 cancer genes. These genomic features were combined using an elastic net regularized logistic regression model for pCR prediction. Multimodal integration was performed using existing methods, and results were compared for early and late fusion strategies. Predictive performance was measured by computing the area under the receiver-operator curve (AUC) metric. Results: Individual data modalities exhibited varying levels of predictive performance for different receptor subtypes. For example, clinical and pathology models performed better for HR+ subtypes, while the genomic prediction model outperformed the rest in the triple negative breast cancer (TNBC) set. Automated DL models using MRI inputs achieved moderate predictive performance on the HR+/HER2-, HR+/HER2+, and HR-/HER2+ subtypes. Subtype-specific models tended to outperform subtype-agnostic models trained in the larger, pan-subtype cohort. End-to-end DL pathology and radiology models that can be deployed and run automatically without human curation to predict pCR directly from images had comparable performance to clinical models using variables curated by expert pathologists and radiologists. Multimodal predictors combining the genomic data with the other three data modalities readily available at initial presentation exhibited the best overall predictive performance, but the gain in predictive performance was small and may not justify the increased requirements in terms of data acquisition and model complexity for certain receptor subtypes. Conclusions: Multimodal data can be incorporated into machine learning models for improved prediction of pCR to NACT in breast cancer. Additional work is needed to validate the clinical utility of these types of multimodal approaches, as well as to determine the most informative data modalities for each hormone receptor subtype and the subset of patients that are most likely to benefit from the use of these models. Automated workflows using DL pre-trained models may provide valuable clinical decision support, guiding escalation and de-escalation therapeutic and monitoring strategies to improve outcomes for pts with high-risk early breast cancer. Citation Format: Sarah Eskreis-Winkler, Francisco Sanchez Vega, Armaan Kohli, Enrico Moiso, Mirella Alto, Chris Fong, Doori Rose, Edaise M. Da Silva, Timothy Dalfonso, David Joon Ho, Anika Begum, Mehnaj Ahmed, Danny Martinez, Andrew Aukerman, Kevin Murphy, Julia An, Mathew Hanna, Jianjiong Gao, Yanis Tazi, Arfath Pasha, Katja Pinker-Domenig, Hong Zhang, Elizabeth Sutton, Sohrab Shah, Pedram Razavi. Multimodal analyses of clinical, radiology, pathology and genomic information for enhanced prediction of response to neoadjuvant therapy in breast cancer [abstract]. In: Proceedings of the San Antonio Breast Cancer Symposium 2024; 2024 Dec 10-13; San Antonio, TX. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(12 Suppl):Abstract nr P1-03-16.