Algometrics: Forecasting Under Algorithmic Feedback
For researchers and practitioners in algorithmic markets (e.g., finance, trading), this work highlights fundamental limitations of standard time-series benchmarks and proposes feedback sensitivity as a necessary additional reporting metric.
The paper introduces algometrics, a framework for time series forecasting under algorithmic feedback, proving that deployment risk is not identifiable from passive historical data, historical model rankings can invert under crowding, and randomized actions can identify short-horizon linear feedback with finite-sample bounds.
In algorithmic markets, predictive models become part of the data-generating process they aim to forecast. Once their outputs are converted into trades, allocations, execution schedules, or risk controls, they change the future data on which they are evaluated. I introduce algometrics, a framework for time series whose evolution depends on the predictive algorithms forecasting them. The framework distinguishes historical risk, measured under passive forecasting, from deployment risk, measured when forecasts drive actions. I prove three results. First, deployment risk is not identifiable from passive historical data alone: even in a one-step linear feedback model, infinitely many algorithm-mediated environments induce the same historical law while implying different deployment risks for the same forecaster. Second, historical model rankings can invert under crowding, so a predictor with lower passive error can have higher deployment error once similar algorithms are adopted. Third, randomized or instrumented actions identify short-horizon linear feedback, and I derive a finite-sample bound for deployment-risk estimation. These results suggest that time-series benchmarks in algorithmic markets should report feedback sensitivity alongside predictive accuracy.