Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agents
This addresses a bottleneck in self-improving agents by enabling effective reuse of past experiences despite context shifts, though it is incremental as it builds on existing trajectory reuse methods.
The paper tackles the problem of context mismatch when autonomous agents reuse successful task trajectories across different tasks, proposing Bifrost, a training-free method that adapts trajectories using context differences, which outperforms existing methods on diverse benchmarks.
Autonomous agents excel in self-improvement through reflection and iterative refinement, which reuse successful task trajectories as in-context examples to assist subsequent reasoning. However, shifting across tasks often introduces a context mismatch. Hence, existing approaches either discard the trajectories or manipulate them using heuristics, leading to a non-negligible fine-tuning cost or unguaranteed performance. To bridge this gap, we reveal a context-trajectory correlation, where shifts of context are highly parallel with shifts of trajectory. Based on this finding, we propose BrIdge contextual gap FoR imprOvised trajectory STeering (Bifrost), a training-free method that leverages context differences to precisely guide the adaptation of previously solved trajectories towards the target task, mitigating the misalignment caused by context shifts. Our trajectory adaptation is conducted at the representation level using agent hidden states, ensuring trajectory transformation accurately aligns with the target context in a shared space. Across diverse benchmarks, Bifrost consistently outperforms existing trajectory reuse and finetuned self-improvement methods, demonstrating that agents can effectively leverage past experiences despite substantial context shifts.