Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions
This work addresses the problem of ensuring stable regression models for out-of-distribution inferences in machine learning, though it appears incremental as it builds on existing manifold theory.
The paper investigates how the extrinsic geometry of a data manifold, specifically its curvature, affects the uniqueness of linear regression solutions, finding that flat dimensions lead to non-unique solutions while curvature influences stability in normal directions.
In this paper, we study linear regression applied to data structured on a manifold. We assume that the data manifold is smooth and is embedded in a Euclidean space, and our objective is to reveal the impact of the data manifold's extrinsic geometry on the regression. Specifically, we analyze the impact of the manifold's curvatures (or higher order nonlinearity in the parameterization when the curvatures are locally zero) on the uniqueness of the regression solution. Our findings suggest that the corresponding linear regression does not have a unique solution when the embedded submanifold is flat in some dimensions. Otherwise, the manifold's curvature (or higher order nonlinearity in the embedding) may contribute significantly, particularly in the solution associated with the normal directions of the manifold. Our findings thus reveal the role of data manifold geometry in ensuring the stability of regression models for out-of-distribution inferences.