Flow to Learn: Flow Matching on Neural Network Parameters
This addresses the problem of adapting image models to new tasks during inference, which is incremental compared to existing work in language models.
The paper tackles the challenge of meta-learning for images by introducing FLoWN, a flow matching model that generates neural network parameters for different tasks, achieving performance that matches or exceeds baselines on in-distribution tasks and improves out-of-distribution few-shot tasks.
Foundational language models show a remarkable ability to learn new concepts during inference via context data. However, similar work for images lag behind. To address this challenge, we introduce FLoWN, a flow matching model that learns to generate neural network parameters for different tasks. Our approach models the flow on latent space, while conditioning the process on context data. Experiments verify that FLoWN attains various desiderata for a meta-learning model. In addition, it matches or exceeds baselines on in-distribution tasks, provides better initializations for classifier training, and is performant on out-of-distribution few-shot tasks while having a fine-tuning mechanism to improve performance.