Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning?
This addresses the problem of applying LLMs to microelectronic reasoning for EDA practitioners, but appears incremental as it tests existing models with a new adaptation method.
The study investigated using Llama-2-7B as a microelectronic Q&A expert in electronic design automation, evaluating its reasoning and generation capabilities with methods including a novel low-rank knowledge distillation scheme, producing qualitative and quantitative results.
In this work, we present empirical results regarding the feasibility of using offline large language models (LLMs) in the context of electronic design automation (EDA). The goal is to investigate and evaluate a contemporary language model's (Llama-2-7B) ability to function as a microelectronic Q & A expert as well as its reasoning, and generation capabilities in solving microelectronic-related problems. Llama-2-7B was tested across a variety of adaptation methods, including introducing a novel low-rank knowledge distillation (LoRA-KD) scheme. Our experiments produce both qualitative and quantitative results.