Interpretable prediction of drug synergy for breast cancer by random forest with features from Boolean modeling of signaling pathways

Abstract Breast cancer is a complex and challenging disease to treat, and despite progress in combating it, drug resistance remains a significant hindrance. Drug combinations have shown promising results in improving therapeutic outcomes, and many machine learning models have been proposed to identify potential drug combinations. Recently, there has been a growing emphasis on enhancing the interpretability of machine learning models to improve our biological understanding of the drug mechanisms underlying the predictions. In this study, we developed a random forest model using simulated protein activities derived from Boolean modeling of breast cancer signaling pathways as input features. The model demonstrates a moderate Pearson's correlation coefficient of 0.40 between the predicted and experimentally observed synergistic scores, with the area under the curve (AUC) of 0.67. Despite its moderate performance, the model offers insights into the interpretable mechanisms behind its predictions. The model's input features consist solely of the individual protein activities simulated in response to drug treatments. Therefore, theframework allows for the analysis of each protein's contribution to the synergy level of each drug pair, enabling a direct interpretation of the drugs' actions on the signaling networks of breast cancer. We demonstrated the interpretability of our approach byidentifying proteins responsible for drug resistance and sensitivity in specific cell lines. For example, the analysis revealed that the combination of MEK and STAT3 inhibitors exhibits only a moderate synergistic effect on MDA-MB-468 due to the negative contributions of mTORC1 and NF-κB that diminish the efficacy of the drug pair. The model further predicted that hyperactive PTEN would sensitize the cells to the drug pair. Our framework enhances the understanding of drug mechanisms at the level of the signaling pathways, potentially leading to more effective treatment designs.