Framework for Benefit-Based Multiclass Classification

Abstract Health datasets typically comprise of data that are heavily skewed towards the healthy class, thus resulting in classifiers being biased towards this majority class. Due to this imbalance of data, traditional performance metrics, such as accuracy, are not appropriate for evaluating the performance of classifiers with the minority class (disease-affected/unhealthy individuals). In addition, classifiers are trained under the assumption that the costs or benefits associated with different decision outcomes are equal. However, this is usually not the case with health data since it is more important to identify disease affected/unhealthy persons rather than healthy individuals. In this paper we address these problems by examining benefits/costs when evaluating the performance of classifiers. Furthermore, we focus on multiclass classification where the outcome can be one of three or more options. We propose modifications to the Naive Bayes and Logistic Regression algorithms to incorporate costs and benefits for the multiclass scenario as well as compare these to an existing algorithm, hierarchical cost-sensitive kernel logistic regression, and also an adapted hierarchical approach with our cost-benefit based logistic regression model. We demonstrate the effectiveness of all approaches for fetal health classification but the proposed approaches can be applied to any imbalance dataset where benefits and costs are important.