A Statistical View of Column Subset Selection
This work provides a unified statistical framework for dimensionality reduction, benefiting researchers in machine learning and statistics by connecting computer science and statistical approaches.
The paper demonstrates that Column Subset Selection (CSS) and Principal Variables are equivalent and can be framed as maximum likelihood estimation in a semi-parametric model, establishing conditions for consistency in high dimensions and enabling efficient CSS with summary statistics, missing data handling, and subset size selection via hypothesis testing.
We consider the problem of selecting a small subset of representative variables from a large dataset. In the computer science literature, this dimensionality reduction problem is typically formalized as Column Subset Selection (CSS). Meanwhile, the typical statistical formalization is to find an information-maximizing set of Principal Variables. This paper shows that these two approaches are equivalent, and moreover, both can be viewed as maximum likelihood estimation within a certain semi-parametric model. Within this model, we establish suitable conditions under which the CSS estimate is consistent in high dimensions, specifically in the proportional asymptotic regime where the number of variables over the sample size converges to a constant. Using these connections, we show how to efficiently (1) perform CSS using only summary statistics from the original dataset; (2) perform CSS in the presence of missing and/or censored data; and (3) select the subset size for CSS in a hypothesis testing framework.