Online Infinite-Dimensional Regression: Learning Linear Operators
This provides theoretical foundations for online learning of infinite-dimensional operators, with implications for functional data analysis and kernel methods.
The paper tackles the problem of learning linear operators between infinite-dimensional Hilbert spaces in online settings, showing that operators with bounded p-Schatten norms are learnable while those with bounded operator norms are not, and revealing a separation between uniform convergence and learnability.
We consider the problem of learning linear operators under squared loss between two infinite-dimensional Hilbert spaces in the online setting. We show that the class of linear operators with uniformly bounded $p$-Schatten norm is online learnable for any $p \in [1, \infty)$. On the other hand, we prove an impossibility result by showing that the class of uniformly bounded linear operators with respect to the operator norm is \textit{not} online learnable. Moreover, we show a separation between sequential uniform convergence and online learnability by identifying a class of bounded linear operators that is online learnable but uniform convergence does not hold. Finally, we prove that the impossibility result and the separation between uniform convergence and learnability also hold in the batch setting.