Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation
For researchers working on LLM-based code generation, this work provides a controlled analysis framework to understand feedback-to-plan attribution, though it is incremental as it focuses on a specific domain (CUDA kernels) and does not introduce a new paradigm.
The paper introduces CUDAnalyst, a method for attributing planning decisions to feedback components in self-evolving LLM agents for CUDA kernel generation, showing that explicit planning benefits only with aligned feedback and that structured multi-feedback interactions drive effective planning.
Large language models (LLMs) have shown strong empirical gains as self-evolving agents for CUDA kernel generation, driven by feedback-conditioned planning across generations. However, how planning decisions attribute and combine heterogeneous feedback signals remains opaque. Standard end-to-end ablations fail to resolve this question, as iterative planning amplifies early perturbations and conflates feedback effects with trajectory-dependent drift. We introduce \texttt{CUDAnalyst}, a unified analysis layer for controlled, generation-level attribution of planning decisions to feedback components via trajectory freezing and selective feedback injection. \texttt{CUDAnalyst} enables stable generation-level evaluation and principled coalitional-style attribution of feedback effects and interactions. Our results show that explicit planning is beneficial only when feedback is aligned, that effective planning emerges from structured multi-feedback interactions, and that high-level plans from stronger reasoning models can partially transfer to weaker ones. These trends hold across reference backbones, representative workloads, and reference induction regimes, indicating that the identified feedback-to-plan structure is robust within the controlled axes studied.