Probability-driven scoring functions in combining linear classifiers
This work addresses an incremental improvement for ensemble classification systems using linear classifiers.
The paper tackled the problem of fusing linear classifiers in ensembles by proposing a probability-driven scoring function that uses both measurement and geometrical spaces, and found that under certain conditions, some improvement can be obtained compared to a reference method on benchmark datasets.
Although linear classifiers are one of the oldest methods in machine learning, they are still very popular in the machine learning community. This is due to their low computational complexity and robustness to overfitting. Consequently, linear classifiers are often used as base classifiers of multiple ensemble classification systems. This research is aimed at building a new fusion method dedicated to the ensemble of linear classifiers. The fusion scheme uses both measurement space and geometrical space. Namely, we proposed a probability-driven scoring function which shape depends on the orientation of the decision hyperplanes generated by the base classifiers. The proposed fusion method is compared with the reference method using multiple benchmark datasets taken from the KEEL repository. The comparison is done using multiple quality criteria. The statistical analysis of the obtained results is also performed. The experimental study shows that, under certain conditions, some improvement may be obtained.