Kinematic Tokenization: Optimization-Based Continuous-Time Tokens for Learnable Decision Policies in Noisy Time Series
This addresses the challenge of learnable decision policies in noisy time series for domains like finance, though it is incremental as it builds on existing tokenization methods with a novel continuous-time approach.
The paper tackled the problem of applying Transformers to noisy continuous-time signals by introducing Kinematic Tokenization, an optimization-based continuous-time representation using spline coefficients, and demonstrated that it sustains non-trivial action distributions and stable policies in financial time series under risk-averse asymmetric losses, unlike discrete baselines that collapse to a cash policy.
Transformers are designed for discrete tokens, yet many real-world signals are continuous processes observed through noisy sampling. Discrete tokenizations (raw values, patches, finite differences) can be brittle in low signal-to-noise regimes, especially when downstream objectives impose asymmetric penalties that rationally encourage abstention. We introduce Kinematic Tokenization, an optimization-based continuous-time representation that reconstructs an explicit spline from noisy measurements and tokenizes local spline coefficients (position, velocity, acceleration, jerk). This is applied to financial time series data in the form of asset prices in conjunction with trading volume profiles. Across a multi-asset daily-equity testbed, we use a risk-averse asymmetric classification objective as a stress test for learnability. Under this objective, several discrete baselines collapse to an absorbing cash policy (the Liquidation Equilibrium), whereas the continuous spline tokens sustain calibrated, non-trivial action distributions and stable policies. These results suggest that explicit continuous-time tokens can improve the learnability and calibration of selective decision policies in noisy time series under abstention-inducing losses.