How to get the most out of Twinned Regression Methods
This work addresses regression tasks for machine learning practitioners, but it appears incremental as it builds on existing twinned regression methods.
The paper tackles the problem of improving twinned regression methods, which predict differences between regression targets rather than the targets themselves, by analyzing their components and developing hybrid approaches. The result includes a more accurate and efficient regression method combining twin neural networks with k-nearest neighbor regression, though no concrete performance numbers are provided.
Twinned regression methods are designed to solve the dual problem to the original regression problem, predicting differences between regression targets rather then the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points. We explore different aspects of twinned regression methods: (1) We decompose different steps in twinned regression algorithms and examine their contributions to the final performance, (2) We examine the intrinsic ensemble quality, (3) We combine twin neural network regression with k-nearest neighbor regression to design a more accurate and efficient regression method, and (4) we develop a simplified semi-supervised regression scheme.