Managing the Stochastic: Foundations of Learning in Neuro-Symbolic Systems for Software Engineering
This addresses reliability issues in AI-driven software engineering by offering a framework to manage stochasticity, though it is incremental in applying established software engineering practices to neuro-symbolic systems.
The paper tackled the problem of stochastic failures in AI coding agents by proposing a dual-state architecture that treats LLMs as part of the environment, separating deterministic control from stochastic generation. This approach improved task success rates by up to 66 percentage points for qualified models at 1.2–2.1× baseline computational cost.
Current approaches to AI coding agents appear to blur the lines between the Large Language Model (LLM) and the agent itself, asking the LLM to make decisions best left to deterministic processes. This leads to systems prone to stochastic failures such as gaming unit tests or hallucinating syntax. Drawing on established software engineering practices that provide deterministic frameworks for managing unpredictable processes, this paper proposes setting the control boundary such that the LLM is treated as a component of the environment environment -- preserving its creative stochasticity -- rather than the decision-making agent. A \textbf{Dual-State Architecture} is formalized, separating workflow state (deterministic control flow) from environment state (stochastic generation). \textbf{Atomic Action Pairs} couple generation with verification as indivisible transactions, where \textbf{Guard Functions} act as sensing actions that project probabilistic outputs onto observable workflow state. The framework is validated on three code generation tasks across 13 LLMs (1.3B--15B parameters). For qualified instruction-following models, task success rates improved by up to 66 percentage points at 1.2--2.1$\times$ baseline computational cost. The results suggest that architectural constraints can substitute for parameter scale in achieving reliable code generation.