Towards Trustworthy Multi-Turn LLM Agents via Behavioral Guidance
This addresses reliability issues in LLM-based agents for multi-turn tasks, though it appears incremental as it builds on existing reinforcement learning formalisms.
The paper tackles the problem of unreliable and unverifiable behavior in multi-turn LLM agents by introducing a framework that integrates task profiling, verifiable reasoning, and constraint-compliant generation, resulting in improved trustworthiness through co-evolution during environment interactions.
Large Language Models demonstrate strong reasoning and generation abilities, yet their behavior in multi-turn tasks often lacks reliability and verifiability. We present a task completion framework that enables LLM-based agents to act under explicit behavioral guidance in environments described by reinforcement learning formalisms with defined observation, action, and reward signals. The framework integrates three components: a lightweight task profiler that selects reasoning and generation strategies, a reasoning module that learns verifiable observation - action mappings, and a generation module that enforces constraint-compliant outputs through validation or deterministic synthesis. We show that as the agent interacts with the environment, these components co-evolve, yielding trustworthy behavior.