Efficient Kirszbraun Extension with Applications to Regression
This work addresses regression problems in machine learning with a new theoretical approach, though it appears incremental as it builds on existing extension theorems.
The paper tackles regression between Hilbert spaces by applying Kirszbraun's extension theorem to supervised learning, achieving a quadratic runtime improvement over state-of-the-art methods through a novel multiplicative weight updates scheme.
We introduce a framework for performing regression between two Hilbert spaces. This is done based on Kirszbraun's extension theorem, to the best of our knowledge, the first application of this technique to supervised learning. We analyze the statistical and computational aspects of this method. We decompose this task into two stages: training (which corresponds operationally to smoothing/regularization) and prediction (which is achieved via Kirszbraun extension). Both are solved algorithmically via a novel multiplicative weight updates (MWU) scheme, which, for our problem formulation, achieves a quadratic runtime improvement over the state of the art. Our empirical results indicate a dramatic improvement over standard off-the-shelf solvers in our setting.