Hyperparameter Trajectory Inference with Conditional Lagrangian Optimal Transport
This addresses the need for flexible neural network adaptation in applications like reinforcement learning and regression, though it is incremental as it builds on existing trajectory inference methods.
The paper tackles the problem of adapting neural networks to evolving user preferences post-deployment without expensive retraining by introducing Hyperparameter Trajectory Inference (HTI), which learns how a network's output distribution changes with hyperparameters and constructs a surrogate model; they propose a method based on conditional Lagrangian optimal transport and show it reconstructs outputs better than alternatives.
Neural networks (NNs) often have critical behavioural trade-offs that are set at design time with hyperparameters-such as reward weights in reinforcement learning or quantile targets in regression. Post-deployment, however, user preferences can evolve, making initial settings undesirable, necessitating potentially expensive retraining. To circumvent this, we introduce the task of Hyperparameter Trajectory Inference (HTI): to learn, from observed data, how a NN's conditional output distribution changes with its hyperparameters, and construct a surrogate model that approximates the NN at unobserved hyperparameter settings. HTI requires extending existing trajectory inference approaches to incorporate conditions, exacerbating the challenge of ensuring inferred paths are feasible. We propose an approach based on conditional Lagrangian optimal transport, jointly learning the Lagrangian function governing hyperparameter-induced dynamics along with the associated optimal transport maps and geodesics between observed marginals, which form the surrogate model. We incorporate inductive biases based on the manifold hypothesis and least-action principles into the learned Lagrangian, improving surrogate model feasibility. We empirically demonstrate that our approach reconstructs NN outputs across various hyperparameter spectra better than other alternatives.