Approximation Theory of Convolutional Architectures for Time Series Modelling
This provides foundational insights for choosing architectures in time series applications, though it is incremental as it parallels existing recurrent results.
The paper tackles the problem of understanding the approximation properties of convolutional architectures for time series modelling, deriving results that show efficiency depends on memory and fine structures, leading to a new spectrum-based regularity measure.
We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem. In the recurrent setting, recent results reveal an intricate connection between approximation efficiency and memory structures in the data generation process. In this paper, we derive parallel results for convolutional architectures, with WaveNet being a prime example. Our results reveal that in this new setting, approximation efficiency is not only characterised by memory, but also additional fine structures in the target relationship. This leads to a novel definition of spectrum-based regularity that measures the complexity of temporal relationships under the convolutional approximation scheme. These analyses provide a foundation to understand the differences between architectural choices for time series modelling and can give theoretically grounded guidance for practical applications.