Central limit theorems for the outputs of fully convolutional neural networks with time series input
This work provides foundational theoretical guarantees for time series analysis with neural networks, addressing a gap in the literature, though it is incremental in extending existing tools to this context.
The authors tackled the lack of theoretical understanding of deep learning for time series by proving that outputs of fully convolutional neural networks with global average pooling become asymptotically Gaussian when inputs are from short-range dependent linear processes as the time series length increases, and they proposed a generalization using learnable weighted pooling coefficients.
Deep learning is widely deployed for time series learning tasks such as classification and forecasting. Despite the empirical successes, only little theory has been developed so far in the time series context. In this work, we prove that if the network inputs are generated from short-range dependent linear processes, the outputs of fully convolutional neural networks (FCNs) with global average pooling (GAP) are asymptotically Gaussian and the limit is attained if the length of the observed time series tends to infinity. The proof leverages existing tools from the theoretical time series literature. Based on our theory, we propose a generalization of the GAP layer by considering a global weighted pooling step with slowly varying, learnable coefficients.