HARIRAYA: a Novel Breast Cancer Pseudo-Color Feature for Multimodal Mammogram using Deep Learning

Breast cancer is the leading cancer in the world. Mammogram is a gold standard for detecting breast cancer at earlier screening because of its sensitivity. Standard grayscale mammogram images are used by expert radiologists and Computer Aided-Diagnosis (CAD) systems. Yet, this original x-ray color provides little information to human radiologists and CAD systems to make decision. This binary color code thus affects sensitivity and specificity of prediction and subsequently affects accuracy. In order to enhance classifier models’ performance, this paper proposes a novel feature-level data integration method that combines features from grayscale mammogram and spectrum mammogram based on a deep neural network (DNN), called HARIRAYA. Pseudo-color is generated using spectrum color code to produce Spectrum mammogram from grayscale mammogram. The DNN is trained with three layers: grayscale, false-color and joint feature representation layers. Empirical results show that the multi-modal DNN model has a better performance in the prediction of malignant breast tissue than single-modal DNN using HARIRAYA features.