S2MAM: Semi-supervised Meta Additive Model for Robust Estimation and Variable Selection
For practitioners of semi-supervised learning, this work provides a method that handles redundant or noisy input variables while maintaining interpretability, though it is an incremental improvement over existing manifold regularization approaches.
This paper proposes a semi-supervised meta additive model (S2MAM) that uses bilevel optimization to automatically identify informative variables, update the similarity matrix, and achieve interpretable predictions, addressing issues with Laplacian regularization in manifold-based semi-supervised learning. Experiments on 4 synthetic and 12 real-world datasets demonstrate robustness and interpretability.
Semi-supervised learning with manifold regularization is a classical framework for jointly learning from both labeled and unlabeled data, where the key requirement is that the support of the unknown marginal distribution has the geometric structure of a Riemannian manifold. Typically, the Laplace-Beltrami operator-based manifold regularization can be approximated empirically by the Laplacian regularization associated with the entire training data and its corresponding graph Laplacian matrix. However, the graph Laplacian matrix depends heavily on the prespecified similarity metric and may lead to inappropriate penalties when dealing with redundant or noisy input variables. To address the above issues, this paper proposes a new \textit{Semi-Supervised Meta Additive Model (S$^2$MAM) based on a bilevel optimization scheme that automatically identifies informative variables, updates the similarity matrix, and simultaneously achieves interpretable predictions. Theoretical guarantees are provided for S$^2$MAM, including the computing convergence and the statistical generalization bound. Experimental assessments across 4 synthetic and 12 real-world datasets, with varying levels and categories of corruption, validate the robustness and interpretability of the proposed approach.