EnCompass: Enhancing Agent Programming with Search Over Program Execution Paths
This addresses a bottleneck in agent programming for developers by enabling easier experimentation and enhancement of LLM-based agents, though it is incremental as it builds on existing programming models.
The paper tackles the entanglement of core workflow logic and inference-time strategies in LLM-based agent programming by introducing probabilistic angelic nondeterminism (PAN), resulting in the EnCompass framework that allows programmers to improve agent reliability and switch strategies with minimal coding.
We introduce a new approach to agent programming, the development of LLM-based agents. Current approaches to agent programming often entangle two aspects of agent design: the core workflow logic and the inference-time strategy (e.g., tree search). We introduce "probabilistic angelic nondeterminism" ("PAN"), a programming model that disentangles these two concerns, allowing the programmer to describe the agent workflow and independently experiment with different inference-time strategies by simply changing a few inputs. We provide an implementation of PAN in Python as the EnCompass framework, which uses a Python decorator to compile agent workflow programs into a search space. We present three case studies that demonstrate how the framework lets the programmer quickly improve the reliability of an agent and easily switch between different inference-time strategies, all with little additional coding.