Measuring temporal effects of agent knowledge by date-controlled tool use
This addresses reliability issues in LLM agents using web search for knowledge grounding, but it is incremental as it builds on existing methods for temporal analysis.
The paper tackled the problem of how temporal changes in web search knowledge affect LLM agent performance by using date-controlled tools as a stress test, showing that agent behavior varies with search temporality but can be mitigated with model choice and reasoning instructions.
Temporal progression is an integral part of knowledge accumulation and update. Web search is frequently adopted as grounding for agent knowledge, yet an improper configuration affects the quality of the agent's responses. Here, we assess the agent behavior using distinct date-controlled tools (DCTs) as stress test to measure the knowledge variability of large language model (LLM) agents. We demonstrate the temporal effects of an LLM agent as a writing assistant, which uses web search to complete scientific publication abstracts. We show that the temporality of search engine translates into tool-dependent agent performance but can be alleviated with base model choice and explicit reasoning instructions such as chain-of-thought prompting. Our results indicate that agent design and evaluations should take a dynamical view and implement measures to account for the temporal influence of external resources to ensure reliability.