Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models
For researchers building foundation models on wireless traces, this work identifies protocol-aware tokenization as the dominant factor, guiding future design choices.
The paper investigates the relative importance of tokenization versus architecture for wireless packet foundation models, finding that protocol-aware tokenization is the primary performance lever (32-point accuracy swing) while architecture choice is a secondary deployment knob (2-point swing). PLUME-DEEP (24-layer GPT) achieves 98.2% top-1 accuracy, and PLUME-MAMBA (Mamba-2) achieves 96.1% with 1.7x higher throughput and 2x longer context.
What matters more for building foundation models for wireless packet traces: the tokenizer or the architecture or both? To answer this question, we build on PLUME Anonymous [2026], which introduced protocol-aware tokenization for 802.11 traces; we scale model depth and transfer the same tokenizer to a fundamentally different architecture family. A deeper GPT (PLUME-DEEP, 24 layers) reaches 98.2% top-1 accuracy, gaining 32 points over the original 12-layer design, while a Mamba-2 state-space variant (PLUME-MAMBA) achieves 96.1% with 1.7x higher throughput and 2x longer context. The key insight emerges from a controlled 2x2 comparison across tokenizers and architectures: changing the tokenizer swings accuracy by 32 points; changing the architecture moves it by only 2. Protocol-aware tokenization is the primary performance lever, and the backbone becomes a deployment knob trading accuracy for speed.