Learning to Generate Unit Test via Adversarial Reinforcement Learning
This addresses the challenge of automating comprehensive unit test generation for programmers and LLM developers, though it is an incremental improvement over existing methods.
The paper tackles the problem of training large language models (LLMs) to generate high-quality unit tests for programming, proposing UTRL, an adversarial reinforcement learning framework that iteratively trains a test generator and a code generator. The result shows that UTRL-trained Qwen3-4B outperforms supervised fine-tuning and frontier models like GPT-4.1 in test quality, with evaluations aligning more closely to ground-truth tests.
Unit testing is a core practice in programming, enabling systematic evaluation of programs produced by human developers or large language models (LLMs). Given the challenges in writing comprehensive unit tests, LLMs have been employed to automate test generation, yet methods for training LLMs to produce high-quality tests remain underexplored. In this work, we propose UTRL, a novel reinforcement learning framework that trains an LLM to generate high-quality unit tests given a programming instruction. Our key idea is to iteratively train two LLMs, the unit test generator and the code generator, in an adversarial manner via reinforcement learning. The unit test generator is trained to maximize a discrimination reward, which reflects its ability to produce tests that expose faults in the code generator's solutions, and the code generator is trained to maximize a code reward, which reflects its ability to produce solutions that pass the unit tests generated by the test generator. In our experiments, we demonstrate that unit tests generated by Qwen3-4B trained via UTRL show higher quality compared to unit tests generated by the same model trained via supervised fine-tuning on human-written ground-truth unit tests, yielding code evaluations that more closely align with those induced by the ground-truth tests. Moreover, Qwen3-4B trained with UTRL outperforms frontier models such as GPT-4.1 in generating high-quality unit tests, highlighting the effectiveness of UTRL in training LLMs for this task.