From Reasoning to Code: GRPO Optimization for Underrepresented Languages
This addresses the problem of code generation for underrepresented programming languages, offering a generalizable approach that could benefit languages with scarce resources, though it appears incremental as it builds on existing methods like GRPO.
The paper tackles the challenge of generating accurate and executable code with LLMs for underrepresented languages like Prolog, which have limited training data, by introducing a method using small-scale Qwen 2.5 models with GRPO to integrate reasoning steps, resulting in significant improvements in reasoning quality, code accuracy, and logical correctness.
Generating accurate and executable code using large language models (LLMs) is challenging for languages with limited public training data compared to popular languages such as Python. This paper introduces a generalizable approach that uses small-scale code versions of the Qwen 2.5 model combined with Group Relative Policy Optimization (GRPO) to enable effective code generation through explicit reasoning steps, which is particularly beneficial for languages with smaller source code databases. Using Prolog as a representative use case -- given its limited online presence -- the initial model faced challenges in generating executable code. After some training steps, the model successfully produces logically consistent and syntactically accurate code by directly integrating reasoning-driven feedback into the reinforcement learning loop. Experimental evaluations using mathematical logic problem benchmarks illustrate significant improvements in reasoning quality, code accuracy, and logical correctness, underscoring the potential of this approach to benefit a wide range of programming languages lacking extensive training resources.