Automatic Programming: Large Language Models and Beyond
It addresses trust and deployment issues for organizations using AI-generated code, but is incremental in proposing existing repair techniques.
The paper examines challenges in deploying automatically generated code from Large Language Models, focusing on quality, security, and programmer responsibility, and suggests that software engineering advances like program repair can enhance code assurance.
Automatic programming has seen increasing popularity due to the emergence of tools like GitHub Copilot which rely on Large Language Models (LLMs). At the same time, automatically generated code faces challenges during deployment due to concerns around quality and trust. In this article, we study automated coding in a general sense and study the concerns around code quality, security and related issues of programmer responsibility. These are key issues for organizations while deciding on the usage of automatically generated code. We discuss how advances in software engineering such as program repair and analysis can enable automatic programming. We conclude with a forward looking view, focusing on the programming environment of the near future, where programmers may need to switch to different roles to fully utilize the power of automatic programming. Automated repair of automatically generated programs from LLMs, can help produce higher assurance code from LLMs, along with evidence of assurance