Conceptually Diverse Base Model Selection for Meta-Learners in Concept Drifting Data Streams
This work addresses the challenge of efficient base model selection for online meta-learners in dynamic environments, representing an incremental improvement with domain-specific applications.
The paper tackles the problem of selecting diverse base models for meta-learners in concept-drifting data streams by proposing a novel approach using Principal Angles to estimate conceptual similarity, with results showing comparable predictive performance to existing methods but with reduced computational overhead, achieving similar performance without parameterization in some cases.
Meta-learners and ensembles aim to combine a set of relevant yet diverse base models to improve predictive performance. However, determining an appropriate set of base models is challenging, especially in online environments where the underlying distribution of data can change over time. In this paper, we present a novel approach for estimating the conceptual similarity of base models, which is calculated using the Principal Angles (PAs) between their underlying subspaces. We propose two methods that use conceptual similarity as a metric to obtain a relevant yet diverse subset of base models: (i) parameterised threshold culling and (ii) parameterless conceptual clustering. We evaluate these methods against thresholding using common ensemble pruning metrics, namely predictive performance and Mutual Information (MI), in the context of online Transfer Learning (TL), using both synthetic and real-world data. Our results show that conceptual similarity thresholding has a reduced computational overhead, and yet yields comparable predictive performance to thresholding using predictive performance and MI. Furthermore, conceptual clustering achieves similar predictive performances without requiring parameterisation, and achieves this with lower computational overhead than thresholding using predictive performance and MI when the number of base models becomes large.