Framework for Benefit-Based Multiclass Classification of Diseases

Abstract Purpose: Health datasets typically comprise of data that are heavily skewed towards the healthy class, thus resulting in classifiers erring towards this majority class. Due to this imbalance of data, traditional performance metrics, such as accuracy, are not appropriate for evaluating the performance of classifiers with the minority class (disease-affected). In addition, classifiers are trained under the assumption that the costs or benefits associated with different decision outcomes are equal. However, this is usually not the case with health data since there are different benefits/costs associated with the correct/incorrect identification of disease affected/unhealthy persons rather than healthy individuals. In this paper we address these problems by examining benefits/costs both when training and evaluating the performance of classifiers. Furthermore,we focus on multiclass classification where the outcome can be one of three or more options. Methods: We propose modifications to the Naive Bayes and Logistic Regression algorithms to incorporate costs and benefits when training for the multiclass scenario, as well as compare these to a recently proposed algorithm in the field, hierarchical cost-sensitive kernel logistic regression, and also an adapted hierarchical approach with our cost-benefit based logistic regression model. Wedemonstrate the effectiveness of all approaches for fetal health classification, vertebral column classification and hepatitis C/fibrosis/cirrhosis prediction. Results: Our proposed multiclass Logistic Regression algorithm outperformed all other algorithms, improving performance with the more critical classes. Conclusion: Our proposed multiclass Logistic Regression algorithm is robust and suitable for cases where costs and benefits of the various decision outcomes are important.