Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance
This addresses the limitation of reactive agents in scenarios requiring foresight, potentially enhancing human-agent collaboration, though it is incremental as it builds on existing LLM agent frameworks.
The paper tackled the problem of developing proactive LLM agents that anticipate and initiate tasks without explicit instructions, by proposing a data-driven approach that fine-tuned models on a new dataset, achieving an F1-Score of 66.47% in proactively offering assistance and outperforming existing models.
Agents powered by large language models have shown remarkable abilities in solving complex tasks. However, most agent systems remain reactive, limiting their effectiveness in scenarios requiring foresight and autonomous decision-making. In this paper, we tackle the challenge of developing proactive agents capable of anticipating and initiating tasks without explicit human instructions. We propose a novel data-driven approach for this problem. Firstly, we collect real-world human activities to generate proactive task predictions. These predictions are then labeled by human annotators as either accepted or rejected. The labeled data is used to train a reward model that simulates human judgment and serves as an automatic evaluator of the proactiveness of LLM agents. Building on this, we develop a comprehensive data generation pipeline to create a diverse dataset, ProactiveBench, containing 6,790 events. Finally, we demonstrate that fine-tuning models with the proposed ProactiveBench can significantly elicit the proactiveness of LLM agents. Experimental results show that our fine-tuned model achieves an F1-Score of 66.47% in proactively offering assistance, outperforming all open-source and close-source models. These results highlight the potential of our method in creating more proactive and effective agent systems, paving the way for future advancements in human-agent collaboration.