Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoning
For researchers working on LLM-based time series analysis, this benchmark fills the gap of evaluating models on irregular time series data, which is common in real-world deployments.
The paper introduces IRTS-ToolBench, a benchmark of 1,700 questions for irregular time series question answering, addressing the gap in evaluating LLMs and AI agents on real-world irregular data. The benchmark spans 10 task types across 13 domains and provides standardized inputs and a reproducible evaluation protocol.
Time series data in real-world deployments is overwhelmingly irregular. Observations are asynchronous, missing values are informative rather than random, and sampling frequencies vary across sensors and operational windows. However, existing Time Series Question Answering (TSQA) benchmarks mostly assume regularly sampled inputs, leaving a fundamental gap in understanding how large language models (LLMs) and AI agents perform under irregular conditions. To bridge this gap, we introduce IRTS-ToolBench, a benchmark of 1,700 questions spanning 10 task types across 13 domains. IRTS-ToolBench is designed to be used independently by any researcher working on LLM-based irregular time series analysis, providing standardized inputs and a reproducible evaluation protocol. Code can be found in https://github.com/SanhornC/IRTS-ToolBench.