Fine-Tuning is Subgraph Search: A New Lens on Learning Dynamics
This work provides a new analytical method for fine-tuning dynamics, which is incremental as it builds on existing mechanistic interpretability concepts to explore learning processes.
The paper tackles the problem of understanding learning dynamics in neural networks by proposing a fine-tuning method that views models as computational graphs and fine-tuning as subgraph search, resulting in a new algorithm called circuit-tuning that balances target task performance and general capabilities.
The study of mechanistic interpretability aims to reverse-engineer a model to explain its behaviors. While recent studies have focused on the static mechanism of a certain behavior, the learning dynamics inside a model remain to be explored. In this work, we develop a fine-tuning method for analyzing the mechanism behind learning. Inspired by the concept of intrinsic dimension, we view a model as a computational graph with redundancy for a specific task, and treat the fine-tuning process as a search for and optimization of a subgraph within this graph. Based on this hypothesis, we propose circuit-tuning, an algorithm that iteratively builds the subgraph for a specific task and updates the relevant parameters in a heuristic way. We first validate our hypothesis through a carefully designed experiment and provide a detailed analysis of the learning dynamics during fine-tuning. Subsequently, we conduct experiments on more complex tasks, demonstrating that circuit-tuning could strike a balance between the performance on the target task and the general capabilities. Our work offers a new analytical method for the dynamics of fine-tuning, provides new findings on the mechanisms behind the training process, and inspires the design of superior algorithms for the training of neural networks.