Multi-Label Learning Through Label-Specific Features with Entropy Guided Clustering

Multi-label learning deals with the problem where each instance is associated with multiple labels. Some methods improve the performance through generating label-specific features rather than inducing model on the original feature space. LIFT [1] first conducts clustering analysis to generate label-specific features, and then some methods are presented to improve the clustering process, e.g. 1) guiding clustering with label entropy, and 2) incorporating clustering ensemble with label similarity. However, these works only focus on label space and ignores the information of instance distribution. To address this issue, we propose a novel method named LIFTED, which systematically employs information theory to guide clustering and clustering ensemble. First, a novel scheme of label entropy is defined on both feature and label spaces, which precisely measures the amount of information of a multi-label dataset given a label. Second, for distinct label, an objective function is established to determine the degree of clustering through minimizing the label entropy. Third, an entropy-based label similarity is designed to guide clustering ensemble, which enhances the model stability and its ability on label correlation exploiting. Experiments on 12 benchmark datasets verify the competitive performance of LIFT against the state-of-art algorithms.