A Trustable Spine Abnormalities Classification System Using ResNet50 and VGG16 Supported by Explainable Artificial Intelligence

Deep learning has been applied in various fields and has been proven to provide good results for classification tasks. However, there is limited understanding of a deep learning model’s decisions, so deep learning is commonly described as a black box. Applying deep learning for critical applications such as medical diagnostic process introduces trust issues. For the deep learning model to be trusted by the medical practitioners, the methods employed by the deep learning model must be seen to be aligned with the diagnostic process employed by the medical practitioners. Explainable methods such as Grad-CAM can be applied to improve the explainability of the deep learning models by providing an visual interpretation of the deep learning classification result decision process. In this study, two deep learning models, VGG16 and ResNet50 are trained using three training methods, one with randomly initialized weights, and two transfer learning methods, which are feature extraction and fine-tuning, to classify the spinal abnormalities based on X-ray images. The classification metrics results are compared and further analyses using Grad-CAM heatmaps are included. The models also evaluated using a stratified five-fold cross-validation, results revealed some disparity between the model’s accuracy and clinical relevance. The randomly initialized VGG16 obtained a classification accuracy of 93.79% but does not focus on clinically relevant regions. On the other hand, not only do the fine-tuned ResNet50 and VGG16 obtain high accuracies of 98.22% and 99.12%, but the heatmaps show that the models focus on more relevant regions. A comparison of the two models shows that the heatmaps produced by the fine-tuned ResNet50 are in more agreement with the clinical view than the fine-tuned VGG16. This study provides a useful reference for interpreting a deep learning-based classification result using explainable method particularly in spine abnormalities analysis with Grad-CAM.