Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks
This work provides foundational theoretical insights for researchers in machine learning and signal processing, though it is incremental as it builds on existing approximation theory.
The paper tackled the problem of understanding the approximation capabilities of temporal convolutional networks for modeling sequences, proving both forward and inverse approximation theorems that characterize the types of sequential relationships these architectures can efficiently capture, with the forward estimate improving upon prior work and the inverse theorem being new.
We present a theoretical analysis of the approximation properties of convolutional architectures when applied to the modeling of temporal sequences. Specifically, we prove an approximation rate estimate (Jackson-type result) and an inverse approximation theorem (Bernstein-type result), which together provide a comprehensive characterization of the types of sequential relationships that can be efficiently captured by a temporal convolutional architecture. The rate estimate improves upon a previous result via the introduction of a refined complexity measure, whereas the inverse approximation theorem is new.