Ensemble Pruning based on Objection Maximization with a General Distributed Framework
This work addresses the time and space efficiency issues in ensemble learning for machine learning practitioners, but it is incremental as it builds on existing ensemble pruning methods with a focus on distributed optimization.
The paper tackles the ensemble pruning problem by formalizing it as an objection maximization problem based on information entropy to balance accuracy and diversity, and proposes a centralized method, a distributed version to speed it up, and a general distributed framework that achieves less time consumption without much accuracy degradation, with experimental results showing a remarkable improvement in execution speed and gratifying accuracy performance.
Ensemble pruning, selecting a subset of individual learners from an original ensemble, alleviates the deficiencies of ensemble learning on the cost of time and space. Accuracy and diversity serve as two crucial factors while they usually conflict with each other. To balance both of them, we formalize the ensemble pruning problem as an objection maximization problem based on information entropy. Then we propose an ensemble pruning method including a centralized version and a distributed version, in which the latter is to speed up the former. At last, we extract a general distributed framework for ensemble pruning, which can be widely suitable for most of the existing ensemble pruning methods and achieve less time consuming without much accuracy degradation. Experimental results validate the efficiency of our framework and methods, particularly concerning a remarkable improvement of the execution speed, accompanied by gratifying accuracy performance.