Fundamental Limitations in Sequential Prediction and Recursive Algorithms: $\mathcal{L}_{p}$ Bounds via an Entropic Analysis
This work addresses foundational theoretical limits in machine learning and signal processing, with incremental contributions to existing entropy-based frameworks.
The paper tackles fundamental limitations in sequential prediction and recursive algorithms by deriving Lp bounds through entropic analysis, with results quantified in terms of conditional entropy and conditions for achieving these bounds explored from an innovations perspective.
In this paper, we obtain fundamental $\mathcal{L}_{p}$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic relationships of the data and/or noises involved, and the derived lower bounds may all be quantified in a conditional entropy characterization. We also study the conditions to achieve the generic bounds from an innovations' viewpoint.