Lossless Compression: A New Benchmark for Time Series Model Evaluation
This provides a principled, information-theoretic evaluation method for time series modeling, addressing a gap in assessing full generative distribution capture, though it is incremental as it builds on existing models and benchmarks.
The paper tackles the problem of evaluating time series models by introducing lossless compression as a new benchmark, based on Shannon's source coding theorem, and shows that it reveals distributional weaknesses in state-of-the-art models like TimeXer, iTransformer, and PatchTST that are missed by traditional tasks.
The evaluation of time series models has traditionally focused on four canonical tasks: forecasting, imputation, anomaly detection, and classification. While these tasks have driven significant progress, they primarily assess task-specific performance and do not rigorously measure whether a model captures the full generative distribution of the data. We introduce lossless compression as a new paradigm for evaluating time series models, grounded in Shannon's source coding theorem. This perspective establishes a direct equivalence between optimal compression length and the negative log-likelihood, providing a strict and unified information-theoretic criterion for modeling capacity. Then We define a standardized evaluation protocol and metrics. We further propose and open-source a comprehensive evaluation framework TSCom-Bench, which enables the rapid adaptation of time series models as backbones for lossless compression. Experiments across diverse datasets on state-of-the-art models, including TimeXer, iTransformer, and PatchTST, demonstrate that compression reveals distributional weaknesses overlooked by classic benchmarks. These findings position lossless compression as a principled task that complements and extends existing evaluation for time series modeling.