Multimodal Forecasting for Commodity Prices Using Spectrogram-Based and Time Series Representations
For practitioners in financial forecasting, SEMF offers a novel method that leverages multimodal fusion to improve prediction accuracy, though the gains are incremental over existing approaches.
The paper introduces Spectrogram-Enhanced Multimodal Fusion (SEMF), which combines spectrogram-based and time series representations for commodity price forecasting, achieving consistent improvements over seven baselines across multiple horizons and metrics.
Forecasting multivariate time series remains challenging due to complex cross-variable dependencies and the presence of heterogeneous external influences. This paper presents Spectrogram-Enhanced Multimodal Fusion (SEMF), which combines spectral and temporal representations for more accurate and robust forecasting. The target time series is transformed into Morlet wavelet spectrograms, from which a Vision Transformer encoder extracts localized, frequency-aware features. In parallel, exogenous variables, such as financial indicators and macroeconomic signals, are encoded via a Transformer to capture temporal dependencies and multivariate dynamics. A bidirectional cross-attention module integrates these modalities into a unified representation that preserves distinct signal characteristics while modeling cross-modal correlations. Applied to multiple commodity price forecasting tasks, SEMF achieves consistent improvements over seven competitive baselines across multiple forecasting horizons and evaluation metrics. These results demonstrate the effectiveness of multimodal fusion and spectrogram-based encoding in capturing multi-scale patterns within complex financial time series.