MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU
This work addresses the challenge of generating efficient native GPU kernels from high-level tensor programs, a critical bottleneck for deploying LLMs on emerging accelerators like Moore Threads GPUs.
MusaCoder introduces a full-stack training framework for native GPU kernel generation on CUDA and MUSA backends, combining progressive data synthesis, rejection fine-tuning, and execution-feedback reinforcement learning. The 9B model matches or exceeds frontier closed-source models, and the 27B model achieves state-of-the-art results on KernelBench benchmarks.
Native GPU kernel generation turns high-level tensor programs into executable, efficient low-level code. Existing Large Language Models (LLMs) struggle with this task, while execution-based reinforcement learning suffers from sparse rewards, reward hacking, and training instability. We present MusaCoder, a full-stack training framework for native GPU kernel generation on CUDA and MUSA backends. MusaCoder combines progressive kernel-oriented data synthesis, diversity-preserving rejection fine-tuning, and execution-feedback Reinforcement Learning (RL) through MooreEval, a distributed verifier and reward environment. To stabilize RL, MusaCoder introduces PrimeEcho for first-turn-anchored multi-turn rewards, Buffered Dynamic Retry for recovering signals from all-failed hard samples, and MirrorPop for off-policy sequence filtering. Experiments on KernelBench and a MUSA-ported variant show that MusaCoder outperforms strong open-source and proprietary baselines in both correctness and empirical speedup, with the 9B model matching or exceeding frontier closed-source models and the 27B model establishing a new state of the art. These results demonstrate not only the effectiveness of full-stack execution-feedback training for native kernel generation, but also the capability of Moore Threads GPUs to support the complete LLM post-training stack, providing a practical foundation for large-model training and optimization on emerging accelerators.