Integrated analysis of multimodal bladder cancer data based on CT, Whole Slide Image and Transcriptome

Abstract In this study, we aim to determine the multimodal data associations between radiological, pathologic, and molecular characteristics in bladder cancer. Furthermore, we tried to find the differential features between high- and low-grade bladder cancer, and improve the prediction accuracy of molecular subtype by multimodal feature fusion. 127 patients with bladder cancer are analyzed in this study. Multimodal associations analysis of CT and pathologic WSI were performed. Structural equation modeling (SEM) was used to measure the structural relationships between multimodal data. Shannon entropy was calculated to evaluate the heterogeneity of bladder cancer. A convolutional neural network was constructed for molecular subtyping based on multimodal features. Feature contribution in model decision-making was explored by discriminative localization method. Cox regression and Kaplan-Meier survival analysis were used to explore the relevance of multimodal features to the prognosis. There are extensive associations between CT and WSI features and parts of these features were related to prognosis. 77 densely associated blocks of feature pairs between CT and WSI were identified. The first block were confirmed with relation to tumor grade. Block 2 and 3 reflect the association of plasm and nucleus of cancer cells with CT features, respectively. SEM detected significant relation between pathological features and molecular subtype. High-grade bladder cancer showed heterogeneity of significance across different scales and higher disorder in microscopic level. Entropy level of WSI can be used to predict the prognosis of patients. For the molecular subtype prediction, an AUC of 0.95±0.09 was achieved based on multimodal feature fusion. The fused CT and WSI features achieved a more accurate molecular subtype prediction. At last, 13 prognosis-related features from CT and WSI were identified.