Static PTX Metrics Track Structural Kernel Regressions but Miss Semantic Ones
For GPU kernel developers and compiler engineers, the paper provides a practical pre-filter to identify structural regressions via static PTX metrics, but notes its limitation for semantic bugs and the unreliability of runtime deltas at small scales.
The paper shows that static PTX metrics (registers, spills, instruction count) reliably detect structural kernel bugs (e.g., missing operations) across five GPU classes, but are blind to semantic bugs (e.g., constant swaps) that produce byte-identical PTX. Measured runtime deltas are hardware-dependent and noisy at sub-millisecond scales, making static PTX deltas a portable pre-filter for distinguishing structural from semantic changes.
We pair each GPU kernel's static PTX metrics (registers, spills, instruction count) with CUDA-event-timed runtime on five GPU classes: RTX 3060, A10, L40S, A100 SXM4, and H100 NVL. In this corpus and toolchain the static and measured signals separate cleanly along one axis. Per-pair Delta-regs and Delta-instrs are identical across all five GPUs for any given (correct, buggy) pair. Measured Delta-perf% is not. Structural bugs that change the kernel's work are unambiguous in the static signal. The gelu_triton_buggy variant, which drops a leading 0.5 factor, removes 8 instructions and 8 registers. The corresponding measured Delta-perf% on RTX 3060 is +3.2%, within the run-to-run noise band at the sub-millisecond scale these corpus kernels occupy. Semantic bugs that swap one constant for another are invisible to the static signal. The softmax_triton_buggy variant, which substitutes other=0.0 for -inf on the masked load, compiles to byte-identical PTX. The paper's bounded claim is that, for this corpus and toolchain, a static-PTX delta gate is a portable pre-filter that separates structural from semantic changes; measured runtime deltas at this scale are hardware- and noise-sensitive and are not a substitute.