Input Conditioned Graph Generation for Language Agents
This work addresses the need for more adaptive and efficient language agents in AI applications, though it is incremental as it builds on existing graph-based frameworks.
The paper tackles the problem of static, handcrafted designs in language agents by developing a method to generate dynamic communication edges based on input, using a pretrained LLM fine-tuned with reinforcement learning. It surpasses previous static approaches by nearly 6% accuracy on a combined MMLU and CMMLU dataset and over 10% with sparsity-inducing loss.
Recent progress in Large Language Models (LLMs) and language agents has demonstrated significant promise for various future applications across multiple disciplines. While traditional approaches to language agents often rely on fixed, handcrafted designs, our research aims to develop both learnable and dynamic agents. Our method uses an existing framework that abstracts language agents as graphs. Within this graph framework, we aim to learn a model that can generate edges for every given input to the language agent. This allows us to generate edges that represent the flow of communication within the graph based on the given input, thereby adjusting the internal communication of a language agent. We learn to generate these edges using a pretrained LLM that is fine-tuned with reinforcement learning. This LLM can be fine-tuned on several datasets simultaneously, and we hypothesize that the model learns to adapt to these different domains during training, achieving good overall performance when encountering data from different domains during deployment. We demonstrate that our approach surpasses the previous static approach by nearly 6% accuracy on a combined dataset of MMLU and CMMLU, and by more than 10% when trained with a sparsity-inducing loss. It also performs superior in additional experiments conducted with the MMLU and Mini Crossword Puzzles datasets. The code is available at https://github.com/lukasVierling/DynamicGPTSwarm.