Pneumonia and tuberculosis detection with chest x-ray images and medical records using deep learning techniques

Pneumonia and tuberculosis are the major public health problems worldwide. These diseases affect the lungs, and if they are not diagnosed properly in time, they can become a fatal health problem. Chest x-ray images are widely used to detect and diagnose Pneumonia and Tuberculosis disease. Detection of Pneumonia and Tuberculosis from chest x-ray images is difficult and requires experience due to the similar pathological features of the diseases. Sometimes a misdiagnosis of the disease occurs due to this similarity. Several researchers used deep learning and machine learning techniques to solve this misdiagnosis problem. However, these studies used the chest x-ray images only to develop Pneumonia and Tuberculosis disease detection models. But using the chest x-ray images alone cannot necessarily lead to accurate disease detection and classification. In the traditional or manual approach, medical records are required to support and correctly interpret the chest x-ray images in the appropriate clinical context. This study develops a multi-input Pneumonia and Tuberculosis detection model using chest x-ray images and medical records to follow the clinical procedure. The study applied a Convolutional Neural Network for the chest x-ray image data and a Multilayer perceptron for the medical record data to develop the models. We implemented feature-level concatenation to join the output feature vectors from the Convolutional Neural Network and a Multilayer perceptron for the development of the disease detection model. For the purpose of comparison, we also developed image-only and medical record-only models. Consequently, the image-only model gives an accuracy of 92.68%, the medical record-only model results in 98.72% accuracy, and the combined model accuracy is improved to 99.61%. In general, the study shows that the fusion of the chest x-ray and the medical records leads to better accuracy and is more similar to the clinical approach.