Twin Neural Network Improved k-Nearest Neighbor Regression
This provides a method for improving regression accuracy in data-limited scenarios, though it appears incremental as it builds on existing techniques.
The paper tackles regression by training a twin neural network to predict differences between targets, then ensembling these differences with nearest neighbors as anchors to improve k-nearest neighbor regression. It shows that this algorithm outperforms both neural networks and k-nearest neighbor regression on small to medium-sized data sets.
Twin neural network regression is trained to predict differences between regression targets rather than the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points. Choosing the anchors to be the nearest neighbors of the unknown data point leads to a neural network-based improvement of k-nearest neighbor regression. This algorithm is shown to outperform both neural networks and k-nearest neighbor regression on small to medium-sized data sets.