Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design
This work addresses the problem of automating chip design for hardware engineers by providing an incremental improvement in LLM-assisted HDL generation through hierarchical methods.
The paper tackles the challenge of generating hardware description language (HDL) for complex chip designs using large language models (LLMs) by introducing hierarchical prompting techniques, which enable successful designs for modules that flat prompting fails on, reduce generation time and costs, and allow smaller open-source LLMs to compete with larger proprietary models.
Large Language Models (LLMs) are effective in computer hardware synthesis via hardware description language (HDL) generation. However, LLM-assisted approaches for HDL generation struggle when handling complex tasks. We introduce a suite of hierarchical prompting techniques which facilitate efficient stepwise design methods, and develop a generalizable automation pipeline for the process. To evaluate these techniques, we present a benchmark set of hardware designs which have solutions with or without architectural hierarchy. Using these benchmarks, we compare various open-source and proprietary LLMs, including our own fine-tuned Code Llama-Verilog model. Our hierarchical methods automatically produce successful designs for complex hardware modules that standard flat prompting methods cannot achieve, allowing smaller open-source LLMs to compete with large proprietary models. Hierarchical prompting reduces HDL generation time and yields savings on LLM costs. Our experiments detail which LLMs are capable of which applications, and how to apply hierarchical methods in various modes. We explore case studies of generating complex cores using automatic scripted hierarchical prompts, including the first-ever LLM-designed processor with no human feedback. Tools for the Recurrent Optimization via Machine Editing (ROME) method can be found at https://github.com/ajn313/ROME-LLM