A Random Projection k Nearest Neighbours Ensemble for Classification via Extended Neighbourhood Rule
This work addresses classification tasks by introducing an incremental improvement over existing kNN ensembles, potentially benefiting researchers and practitioners in machine learning.
The paper tackles the problem of improving classification accuracy by proposing a novel ensemble method that combines random projections with an extended neighborhood rule for kNN base learners, achieving enhanced performance through additional randomness and preserved feature information.
Ensembles based on k nearest neighbours (kNN) combine a large number of base learners, each constructed on a sample taken from a given training data. Typical kNN based ensembles determine the k closest observations in the training data bounded to a test sample point by a spherical region to predict its class. In this paper, a novel random projection extended neighbourhood rule (RPExNRule) ensemble is proposed where bootstrap samples from the given training data are randomly projected into lower dimensions for additional randomness in the base models and to preserve features information. It uses the extended neighbourhood rule (ExNRule) to fit kNN as base learners on randomly projected bootstrap samples.