LENS: Large Pre-trained Transformer for Exploring Financial Time Series Regularities
This work addresses the problem of ineffective traditional and pre-training methods for financial time series modeling, offering a domain-specific solution that is incremental in adapting pre-trained models to high-noise environments.
The paper tackles the challenge of modeling financial time series, which have high stochasticity and low signal-to-noise ratios, by proposing LENS, a pre-trained transformer tailored to this domain. Pre-trained on 100 billion observations, LENS achieves exceptional results across various downstream tasks.
Modeling large-scale time series has gained significant attention in recent years. However, its direct application in finance remains challenging due to substantial differences in data characteristics across domains. Specifically, financial systems feature inherent stochasticity and low signal-to-noise ratios, rendering traditional methods and pre-training approaches ineffective. This underscores the urgent need for a foundation model tailored to financial time series. To bridge this gap, we propose \textbf{LENS}, a pre-trained model for this domain. \textbf{LENS} effectively captures the complexity of financial stochastic systems through a carefully crafted model architecture and mitigates noise during pre-training by using an invertible embedding module. We provide a rigorous theoretical explanation of the model's effectiveness and validate its performance through extensive experiments. Pre-trained on a dataset comprising 100 billion financial observations, \textbf{LENS} achieves exceptional results across a wide range of critical downstream tasks. Moreover, our work offers practical insights into developing pre-trained time series models in high-noise environments, paving the way for further advancements in this pivotal research domain.