Liquid Democracy for Low-Cost Ensemble Pruning
This work addresses computational efficiency for machine learning practitioners, though it is incremental as it applies an existing social choice framework to ensemble pruning.
The paper tackles the problem of high computational cost in ensemble training by proposing a method inspired by liquid democracy to prune redundant classifiers, resulting in reduced training costs and higher accuracy than some boosting methods.
We argue that there is a strong connection between ensemble learning and a delegative voting paradigm -- liquid democracy -- that can be leveraged to reduce ensemble training costs. We present an incremental training procedure that identifies and removes redundant classifiers from an ensemble via delegation mechanisms inspired by liquid democracy. Through both analysis and extensive experiments we show that this process greatly reduces the computational cost of training compared to training a full ensemble. By carefully selecting the underlying delegation mechanism, weight centralization in the classifier population is avoided, leading to higher accuracy than some boosting methods. Furthermore, this work serves as an exemplar of how frameworks from computational social choice literature can be applied to problems in nontraditional domains.