Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models
For practitioners building scalable time series foundation models, this work addresses token efficiency bottlenecks by introducing an asymmetric compression method that accelerates inference while maintaining or improving performance.
The paper identifies inefficiencies in uniform token processing for time series language models, where TS tokens have redundant frequency patterns and prompt tokens attenuate with depth. It proposes an adaptive token budgeting framework that compresses TS tokens via frequency-domain structure and reduces prompt tokens across layers, achieving up to 7.68× inference acceleration and performance gains in 78% of evaluated settings across forecasting, classification, imputation, and anomaly detection.
Large language models (LLMs) have enabled time series (TS) analysis by jointly modeling numerical observations and textual context through a shared token interface. However, TS tokens and prompt tokens exhibit fundamentally different information structures, making uniform token processing inefficient. In this paper, we study token efficiency in TS language modeling from an asymmetric-token perspective. We show that TS tokens have highly uneven spectral contributions, where many tokens share redundant frequency patterns while a small subset preserves critical temporal evidence. We also observe that prompt-token influence attenuates with model depth, suggesting that full prompt retention across all layers is unnecessary. Based on these findings, we develop an adaptive token budgeting framework that compresses TS tokens via frequency-domain structure and progressively reduces prompt tokens across layers. Experiments across forecasting, classification, imputation, and anomaly detection demonstrate up to \textit{\textbf{7.68$\times$}} inference acceleration and performance gains in \textit{\textbf{78\%}} of evaluated settings, showing the effectiveness of asymmetric token compression for scalable TS foundation models.