Using a Feedback Loop for LLM-based Infrastructure as Code Generation
This addresses the challenge of infrastructure management for software developers, but the results indicate it is incremental with limited practical impact.
The paper tackled the problem of using LLMs for Infrastructure as Code generation by implementing a feedback loop to improve code based on errors and warnings, finding that the loop's effectiveness decreases exponentially with each iteration until it plateaus and becomes ineffective.
Code generation with Large Language Models (LLMs) has helped to increase software developer productivity in coding tasks, but has yet to have significant impact on the tasks of software developers that surround this code. In particular, the challenge of infrastructure management remains an open question. We investigate the ability of an LLM agent to construct infrastructure using the Infrastructure as Code (IaC) paradigm. We particularly investigate the use of a feedback loop that returns errors and warnings on the generated IaC to allow the LLM agent to improve the code. We find that, for each iteration of the loop, its effectiveness decreases exponentially until it plateaus at a certain point and becomes ineffective.