LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection
This work addresses anomaly detection for applications like finance and healthcare, but it is incremental as it combines existing methods like LLMs, RL, and VAEs in a novel way.
The paper tackled the problem of time series anomaly detection with sparse labels and complex patterns by integrating LLM-based reward shaping, RL, VAE-enhanced scaling, and active learning, achieving state-of-the-art accuracy on Yahoo-A1 and SMD benchmarks under limited labeling budgets.
Detecting anomalies in time series data is crucial for finance, healthcare, sensor networks, and industrial monitoring applications. However, time series anomaly detection often suffers from sparse labels, complex temporal patterns, and costly expert annotation. We propose a unified framework that integrates Large Language Model (LLM)-based potential functions for reward shaping with Reinforcement Learning (RL), Variational Autoencoder (VAE)-enhanced dynamic reward scaling, and active learning with label propagation. An LSTM-based RL agent leverages LLM-derived semantic rewards to guide exploration, while VAE reconstruction errors add unsupervised anomaly signals. Active learning selects the most uncertain samples, and label propagation efficiently expands labeled data. Evaluations on Yahoo-A1 and SMD benchmarks demonstrate that our method achieves state-of-the-art detection accuracy under limited labeling budgets and operates effectively in data-constrained settings. This study highlights the promise of combining LLMs with RL and advanced unsupervised techniques for robust, scalable anomaly detection in real-world applications.