Implicit meta-learning may lead language models to trust more reliable sources
This research addresses the problem of improving model trustworthiness and controllability for AI developers and users, though it is incremental as it builds on existing meta-learning concepts.
The study demonstrates that language models can implicitly learn to prioritize more reliable sources during fine-tuning by using synthetic tags as indicators of usefulness, showing that this effect occurs across different model types and scales, with larger models and smaller batch sizes enhancing the phenomenon.
We demonstrate that LLMs may learn indicators of document usefulness and modulate their updates accordingly. We introduce random strings ("tags") as indicators of usefulness in a synthetic fine-tuning dataset. Fine-tuning on this dataset leads to implicit meta-learning (IML): in further fine-tuning, the model updates to make more use of text that is tagged as useful. We perform a thorough empirical investigation of this phenomenon, finding (among other things) that (i) it occurs in both pretrained LLMs and those trained from scratch, as well as on a vision task, and (ii) larger models and smaller batch sizes tend to give more IML. We also use probing to examine how IML changes the way models store knowledge in their parameters. Finally, we reflect on what our results might imply about capabilities, risks, and controllability of future AI systems. Our code can be found at https://github.com/krasheninnikov/internalization.