SLMFix: Leveraging Small Language Models for Error Fixing with Reinforcement Learning
This addresses code generation reliability for developers using domain-specific languages, but it is incremental as it builds on existing LLM and RL techniques.
The paper tackles the problem of syntactic errors in LLM-generated code, especially for low-resource programming languages, by proposing SLMFix, a pipeline that uses a small language model with reinforcement learning to fix errors, achieving over 95% pass rate on a static validator and outperforming supervised finetuning for 7B models.
Recent advancements in large language models (LLMs) have shown very impressive capabilities in code generation across many programming languages. However, even state-of-the-art LLMs generate programs that contains syntactic errors and fail to complete the given tasks, especially for low-resource programming languages (LRPLs). In addition, high training cost makes finetuning LLMs unaffordable with constrained computational resources, further undermining the effectiveness of LLMs for code generation. In this work, we propose SLMFix, a novel code generation pipeline that leverages a small language model (SLM) finetuned using reinforcement learning (RL) techniques to fix syntactic errors in LLM-generated programs to improve the quality of LLM-generated programs for domain-specific languages (DSLs). In specific, we applied RL on the SLM for the program repair task using a reward calculated using both a static validator and a static semantic similarity metric. Our experimental results demonstrate the effectiveness and generalizability of our approach across multiple DSLs, achieving more than 95% pass rate on the static validator. Notably, SLMFix brings substantial improvement to the base model and outperforms supervised finetuning approach even for 7B models on a LRPL, showing the potential of our approach as an alternative to traditional finetuning approaches.