Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series
This research is significant for financial analysts and machine learning practitioners, as it highlights the importance of considering the optimizer as a source of inductive bias in underspecified settings.
The authors tackled the problem of neural networks achieving similar out-of-sample error in financial time series forecasting, and found that different model-training-pipeline pairs learn qualitatively different functions, resulting in nearly 3x turnover dispersion at comparable Sharpe ratios. The optimizer choice was shown to reshape non-linear response profiles and temporal dependence differently.
Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale volatility forecasting for S$\&$P 500 stocks, we show that different model-training-pipeline pairs with identical test loss learn qualitatively different functions. Across architectures, predictive accuracy remains unchanged, yet optimizer choice reshapes non-linear response profiles and temporal dependence differently. These divergences have material consequences for decisions: volatility-ranked portfolios trace a near-vertical Sharpe-turnover frontier, with nearly $3\times$ turnover dispersion at comparable Sharpe ratios. We conclude that in underspecified settings, optimization acts as a consequential source of inductive bias, thus model evaluation should extend beyond scalar loss to encompass functional and decision-level implications.