Test-Oriented Programming: rethinking coding for the GenAI era
This addresses the problem of reducing developer effort in coding by leveraging LLMs for a higher abstraction level, though it is incremental as it builds on existing LLM capabilities.
The authors propose Test-Oriented Programming (TOP), a new paradigm where developers check test code from natural language specs while LLMs generate production code, and they demonstrate feasibility with a proof-of-concept tool on a small command-line program using two LLMs, yielding promising results.
Large language models (LLMs) have shown astonishing capability of generating software code, leading to its use to support developers in programming. Proposed tools have relied either on assistants for improved auto-complete or multi-agents, in which different model instances are orchestrated to perform parts of a problem to reach a complete solution. We argue that LLMs can enable a higher-level of abstraction, a new paradigm we called Test-Oriented Programming (TOP). Within this paradigm, developers only have to check test code generated based on natural language specifications, rather than focusing on production code, which could be delegated to the LLMs. To evaluate the feasibility of this proposal, we developed a proof-of-concept tool and used it to generate a small command-line program employing two different LLMs. We obtained promising results and identified challenges for the use of this paradigm for real projects.