PROFIT: A Specialized Optimizer for Deep Fine Tuning
This work addresses the need for better fine-tuning performance in generative AI, computer vision, and robotics, though it is incremental as it builds on existing fine-tuning methods.
The paper tackles the problem of fine-tuning pre-trained models for improved performance by introducing PROFIT, a specialized optimizer that uses temporal gradient-orthogonalization to regularize optimization based on converged model properties, outperforming existing methods in tasks like image classification and multimodal language model training.
The fine-tuning of pre-trained models has become ubiquitous in generative AI, computer vision, and robotics. Although much attention has been paid to improving the efficiency of fine-tuning model, there has been less scholarship around fine-tuning specifically for improved model performance. To remedy this gap, we present PROFIT, one of the first optimizers designed to incrementally fine-tune converged models on new tasks and/or datasets. Unlike traditional optimizers such as SGD or Adam, which make minimal assumptions due to random initializations, PROFIT takes the properties of a converged model into account explicitly to regularize the optimization process. Employing a temporal gradient-orthogonalization process, PROFIT outperforms fine-tuning methods in various tasks, from image classification to multimodal language model training to large-scale motion prediction. Moreover, PROFIT is encapsulated as a modular optimizer, which makes it easy to integrate directly into any training pipeline with minimal engineering effort.