Meta-Neighborhoods
This addresses the need for adaptive prediction in general AI, though it appears incremental as an extension of k-nearest-neighbors.
The paper tackles the problem of making adaptive predictions based on input neighborhoods in AI, proposing Meta-Neighborhoods as a semi-parametric method that generalizes k-nearest-neighbors and improves predictive distribution accuracy, with extensive studies showing its superiority.
Making an adaptive prediction based on one's input is an important ability for general artificial intelligence. In this work, we step forward in this direction and propose a semi-parametric method, Meta-Neighborhoods, where predictions are made adaptively to the neighborhood of the input. We show that Meta-Neighborhoods is a generalization of $k$-nearest-neighbors. Due to the simpler manifold structure around a local neighborhood, Meta-Neighborhoods represent the predictive distribution $p(y \mid x)$ more accurately. To reduce memory and computation overhead, we propose induced neighborhoods that summarize the training data into a much smaller dictionary. A meta-learning based training mechanism is then exploited to jointly learn the induced neighborhoods and the model. Extensive studies demonstrate the superiority of our method.