BPE: Behavioral Profiling Ensemble
This work addresses the problem of inefficient and less accurate ensemble learning for machine learning practitioners, representing an incremental improvement over existing dynamic ensemble selection methods.
The paper tackles the limitation of traditional ensemble methods by proposing the Behavioral Profiling Ensemble (BPE) framework, which uses intrinsic behavioral profiles to assign aggregation weights, resulting in improved predictive accuracy and reduced computational overhead on 42 real-world datasets.
In the field of machine learning, ensemble learning is widely recognized as a pivotal strategy for pushing the boundaries of predictive performance. Traditional static ensemble methods typically assign weights by treating each base learner as a whole, thereby overlooking that individual models exhibit varying competence across different regions of the instance space. Dynamic Ensemble Selection (DES) was introduced to address this limitation. However, both static and dynamic approaches predominantly rely on inter-model differences as the basis for integration; this inter-model perspective neglects models' intrinsic characteristics and often requires heavy reliance on reference sets for competence estimation. We propose the Behavioral Profiling Ensemble (BPE) framework, which introduces a model-centric integration paradigm. Unlike traditional methods, BPE constructs an intrinsic behavioral profile $\mathcal{P}_k$ for each model and derives aggregation weights from the deviation between a model's response to a test instance and its established profile; in this work, we instantiate $\mathcal{P}_k$ with entropy-based summary statistics (e.g., mean and variance). Extensive experiments on 42 real-world datasets show that BPE-derived algorithms outperform state-of-the-art DES baselines, increasing predictive accuracy while reducing computational and storage overhead.