Statistical Software Engineering with Tuned Variables
This addresses the challenge of system maintenance for AI engineering teams, but appears incremental by building on existing SE4AI and governed tuning work.
The paper tackles the problem of maintaining AI-enabled systems under changing conditions by proposing to treat model choices and operational thresholds as tuned variables governed through statistical evaluation, though no concrete performance numbers are provided.
The maintained artifact in an AI-enabled system is not code plus settings, but a versioned governed program space: domains, structural constraints, eligibility, evaluation assets, and a statistical release gate. AI-enabled systems operate under changing world conditions: provider models and APIs change, input distributions drift, evaluation sets age, and objectives such as quality, cost, latency, and safety are renegotiated over time. In practice, teams often respond through ad hoc changes to model choice, retrieval policy, prompt structure, and operational thresholds. Fixed-assignment reasoning is therefore insufficient: a chosen assignment is valid only relative to an environment, evaluation set, and policy state. We argue that such choices should be treated as tuned variables: program variables maintained under governance as environments and evaluation sets evolve. Building on SE4AI work and our prior work on governed tuning, this paper positions the governed space as the software-engineering object. Here, statistical means that promotion relies on sampled evaluation sets, estimated evidence, effect-size margins, and confidence/risk thresholds.