LGFeb 27, 2023
A Brief Survey on the Approximation Theory for Sequence Modelling
arXiv:2302.13752v115 citationsh-index: 79
Originality Synthesis-oriented
AI Analysis
This is an incremental survey that organizes existing knowledge for researchers in machine learning theory.
The paper surveys approximation theory developments for sequence modeling in machine learning, focusing on classifying results across model architectures and outlining future research directions.
We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation paradigms, and the insights one can gain from these results. We also outline some future research directions towards building a theory of sequence modelling.