A Structural Approach to Coordinate-Free Statistics
This work provides a foundational extension of statistical methods to abstract spaces, potentially benefiting researchers in probability theory, statistics, and machine learning, though it appears incremental in building on existing theorems.
The authors tackled the problem of learning in general topological vector spaces by extending Ordinary Least Squares (OLS) estimators and the Gauss-Markov theorem to this setting, proving that OLS is a best linear unbiased estimator and constructing a stochastic version for uncorrelated implies independent measures.
We consider the question of learning in general topological vector spaces. By exploiting known (or parametrized) covariance structures, our Main Theorem demonstrates that any continuous linear map corresponds to a certain isomorphism of embedded Hilbert spaces. By inverting this isomorphism and extending continuously, we construct a version of the Ordinary Least Squares estimator in absolute generality. Our Gauss-Markov theorem demonstrates that OLS is a "best linear unbiased estimator", extending the classical result. We construct a stochastic version of the OLS estimator, which is a continuous disintegration exactly for the class of "uncorrelated implies independent" (UII) measures. As a consequence, Gaussian measures always exhibit continuous disintegrations through continuous linear maps, extending a theorem of the first author. Applying this framework to some problems in machine learning, we prove a useful representation theorem for covariance tensors, and show that OLS defines a good kriging predictor for vector-valued arrays on general index spaces. We also construct a support-vector machine classifier in this setting. We hope that our article shines light on some deeper connections between probability theory, statistics and machine learning, and may serve as a point of intersection for these three communities.