Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learning
This work addresses the challenge of automatic hardware design generation for engineers, but it is incremental as it builds on existing LLM methods with a novel reinforcement learning approach.
The paper tackles the problem of generating Verilog code for RTL designs using large language models, which struggle with parallel structures due to limited training data, and introduces VeriSeek, an LLM enhanced by reinforcement learning with code-structure feedback, achieving state-of-the-art performance across multiple benchmarks.
Recent advancements in large language models (LLMs) have sparked significant interest in the automatic generation of Register Transfer Level (RTL) designs, particularly using Verilog. Current research on this topic primarily focuses on pre-training and instruction tuning, but the effectiveness of these methods is constrained by the limited availability of training data, as public Verilog code is far less abundant than software code. In particular, these methods struggle to effectively capture Verilog parallel code structures, which fundamentally differ from the imperative, sequential control flow typical in most software programming languages. This paper introduces VeriSeek, an LLM enhanced by reinforcement learning using a limited amount of high-quality training data to achieve high Verilog code generation performance. Our reinforcement learning approach employs code structure information as feedback signals to refine the pre-trained model, enabling it to effectively learn important patterns from Verilog code with parallel structures. Experiments show that VeriSeek outperforms state-of-the-art methods across multiple benchmarks.