DevFormer: A Symmetric Transformer for Context-Aware Device Placement
This addresses the problem of efficient hardware design optimization for engineers, though it appears incremental as it adapts transformers to a new domain with specific modifications.
The paper tackles the problem of hardware design optimization, specifically decoupling capacitor placement, by introducing DevFormer, a transformer-based architecture that incorporates inductive biases like relative positional embeddings and action-permutation symmetricity to work with limited offline data. The result shows it outperforms state-of-the-art methods in simulated and real hardware, reducing components by over 30% while improving performance.
In this paper, we present DevFormer, a novel transformer-based architecture for addressing the complex and computationally demanding problem of hardware design optimization. Despite the demonstrated efficacy of transformers in domains including natural language processing and computer vision, their use in hardware design has been limited by the scarcity of offline data. Our approach addresses this limitation by introducing strong inductive biases such as relative positional embeddings and action-permutation symmetricity that effectively capture the hardware context and enable efficient design optimization with limited offline data. We apply DevFoemer to the problem of decoupling capacitor placement and show that it outperforms state-of-the-art methods in both simulated and real hardware, leading to improved performances while reducing the number of components by more than $30\%$. Finally, we show that our approach achieves promising results in other offline contextual learning-based combinatorial optimization tasks.