Learning Functional Transduction
This work addresses the need for efficient and specific regression methods in machine learning, particularly for complex physical systems with limited data, though it appears incremental as it combines existing transductive and inductive approaches.
The authors tackled the problem of regression tasks by proposing a hybrid approach that meta-learns transductive regression principles to create efficient in-context neural approximators, resulting in almost instantaneous capture of functional relationships with few examples and reduced computational cost for applications like partial differential equations and climate modeling.
Research in machine learning has polarized into two general approaches for regression tasks: Transductive methods construct estimates directly from available data but are usually problem unspecific. Inductive methods can be much more specific but generally require compute-intensive solution searches. In this work, we propose a hybrid approach and show that transductive regression principles can be meta-learned through gradient descent to form efficient in-context neural approximators by leveraging the theory of vector-valued Reproducing Kernel Banach Spaces (RKBS). We apply this approach to function spaces defined over finite and infinite-dimensional spaces (function-valued operators) and show that once trained, the Transducer can almost instantaneously capture an infinity of functional relationships given a few pairs of input and output examples and return new image estimates. We demonstrate the benefit of our meta-learned transductive approach to model complex physical systems influenced by varying external factors with little data at a fraction of the usual deep learning training computational cost for partial differential equations and climate modeling applications.