Learning Densities Conditional on Many Interacting Features
This addresses a problem in statistics and machine learning for applications requiring full density estimation beyond mean prediction, but it appears incremental as it builds on existing nonparametric and feature selection techniques.
The paper tackles the challenge of learning conditional densities for high-dimensional discrete features with interactions, introducing a nonparametric Bayes method using tensor factorization and multistage feature selection to flexibly model densities, though no concrete numerical results are provided.
Learning a distribution conditional on a set of discrete-valued features is a commonly encountered task. This becomes more challenging with a high-dimensional feature set when there is the possibility of interaction between the features. In addition, many frequently applied techniques consider only prediction of the mean, but the complete conditional density is needed to answer more complex questions. We demonstrate a novel nonparametric Bayes method based upon a tensor factorization of feature-dependent weights for Gaussian kernels. The method makes use of multistage feature selection for dimension reduction. The resulting conditional density morphs flexibly with the selected features.