A Novel ECOC Algorithm with Centroid Distance Based Soft Coding Scheme
This work addresses a specific bottleneck in ECOC coding for multi-class classification, offering an incremental improvement over existing methods.
The paper tackles the problem of information loss in ternary coding within the Error-Correcting Output Codes (ECOC) framework by proposing a Centroid distance-based Soft coding scheme (CSECOC), which uses soft elements to reflect class distribution tendencies and regressors as base learners, achieving comparable or better classification accuracy on five UCI datasets with small ensembles.
In ECOC framework, the ternary coding strategy is widely deployed in coding process. It relabels classes with {"-1,0,1" }, where -1/1 means to assign the corresponding classes to the negative/positive group, and label 0 leads to ignore the corresponding classes in the training process. However, the application of hard labels may lose some information about the tendency of class distributions. Instead, we propose a Centroid distance-based Soft coding scheme to indicate such tendency, named as CSECOC. In our algorithm, Sequential Forward Floating Selection (SFFS) is applied to search an optimal class assignment by minimizing the ratio of intra-group and inter-group distance. In this way, a hard coding matrix is generated initially. Then we propose a measure, named as coverage, to describe the probability of a sample in a class falling to a correct group. The coverage of a class a group replace the corresponding hard element, so as to form a soft coding matrix. Compared with the hard ones, such soft elements can reflect the tendency of a class belonging to positive or negative group. Instead of classifiers, regressors are used as base learners in this algorithm. To the best of our knowledge, it is the first time that soft coding scheme has been proposed. The results on five UCI datasets show that compared with some state-of-art ECOC algorithms, our algorithm can produce comparable or better classification accuracy with small scale ensembles.