Enhancing Multi-Agent Communication through Attention Steering with Context Relevance
Addresses the context dilution problem in multi-agent LLM systems, improving performance for collaborative reasoning tasks.
LLM-based multi-agent systems suffer performance degradation due to irrelevant context accumulation. Agent-Radar, a training-free context management method with temporal and spatial decay, achieves up to 7.64 absolute point gains over SOTA across five benchmarks.
LLM-based multi-agent systems have demonstrated remarkable performance on complex tasks through collaborative reasoning. However, these systems tend to rapidly accumulate extremely long conversation histories during interaction. As conversations lengthen, relevant information is increasingly diluted by irrelevant context, leading to degraded performance. In this work, we present Agent-Radar, a training-free context management method that dynamically steers each agent's attention toward relevant context with a novel temporal and spatial decay mechanism. Our experiments demonstrate that Agent-Radar outperforms state-of-the-art methods across five different benchmarks, yielding gains of up to 7.64 absolute points. Furthermore, our analysis shows that Agent-Radar remains effective and robust as the number of agents and interaction rounds increases. Finally, the ablation study shows that core components in Agent-Radar are crucial to performance and generalizable in different settings.