SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation
For high-energy physics, SPADE provides a competitive generative model for point-cloud calorimeter data, but the mechanism is incremental as it adapts existing transformer architectures.
SPADE introduces a split-and-delay embedding mechanism for autoregressive transformers that independently embeds multiple features per token and delays feature streams to learn intra-token correlations via self-attention. Applied to calorimeter shower simulation, it matches the state-of-the-art AllShowers on photon showers and significantly outperforms OmniJet-αC.
We introduce SPADE (SPlit And Delay Embeddings), an autoregressive transformer for sequences whose tokens carry multiple features. Rather than embedding these features jointly, SPADE embeds them independently. Delaying each feature stream relative to the previous one allows intra-token correlations to be learned by the standard self-attention mechanism. Applied to point-cloud calorimeter shower generation in the highly granular ILD detector, SPADE is competitive with the state of the art AllShowers model on photon showers, and substantially outperforms its VQ-VAE-based predecessor OmniJet-$α_C$. The mechanism is applicable to any generative task with multi-feature tokens, enabling LLM-style pretraining workflows for higher-dimensional data.