CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels
For deep learning practitioners needing efficient and flexible attention kernel implementations, CuBridge bridges the gap between expert-written high-performance kernels and automated adaptation, offering a practical solution for evolving attention mechanisms.
CuBridge is an LLM-based framework that adapts expert-written attention kernels to new variants via a lift-transfer-lower workflow, consistently producing correct kernels and substantially outperforming general frameworks, compiler-based approaches, and prior LLM-based methods across diverse attention variants and GPU platforms.
Efficient CUDA implementations of attention mechanisms are critical to modern deep learning systems, yet supporting diverse and evolving attention variants remains challenging. Existing frameworks and compilers trade performance for flexibility, while expert-written kernels achieve high efficiency but are difficult to adapt. Recent work explores large language models (LLMs) for GPU kernel generation, but prior studies report unstable correctness and significant performance gaps for complex operators such as attention. We present CuBridge, an LLM-based framework that adapts expert-written attention kernels through a structured lift-transfer-lower workflow. CuBridge starts from expert-written CUDA attention kernels and lifts them into an executable intermediate representation that makes execution orchestration explicit while abstracting low-level CUDA syntax. Given a user-provided PyTorch specification, CuBridge generates and verifies a target IR program, then reconstructs optimized CUDA code via reference-guided lowering. Across diverse attention variants and GPU platforms, CuBridge consistently produces correct kernels and substantially outperforms general frameworks, compiler-based approaches, and prior LLM-based methods.