LLM-CSEC: Empirical Evaluation of Security in C/C++ Code Generated by Large Language Models
This addresses security concerns for developers relying on LLM-generated code, though it is incremental as it applies existing static analysis to new LLM outputs.
The study evaluated the security of C/C++ code generated by ten large language models, finding a concerning amount of Common Weakness Enumeration vulnerabilities, which highlights risks for developers using such code.
The security of code generated by large language models (LLMs) is a significant concern, as studies indicate that such code often contains vulnerabilities and lacks essential defensive programming constructs. This work focuses on examining and evaluating the security of LLM-generated code, particularly in the context of C/C++. We categorized known vulnerabilities using the Common Weakness Enumeration (CWE) and, to study their criticality, mapped them to CVEs. We used ten different LLMs for code generation and analyzed the outputs through static analysis. The amount of CWEs present in AI-generated code is concerning. Our findings highlight the need for developers to be cautious when using LLM-generated code. This study provides valuable insights to advance automated code generation and encourage further research in this domain.