Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation
This addresses the curse of dimensionality in generative modeling for machine learning applications, offering a non-parametric alternative to predefined priors in methods like variational auto-encoders, though it appears incremental as it builds on existing auto-encoder and tensor concepts.
The paper tackles the challenge of learning generative probabilistic models in high dimensions by proposing a joint framework for dimensionality reduction and non-parametric density estimation, achieving promising results on toy, tabular, and image datasets for tasks like regression, sampling, and anomaly detection.
Learning generative probabilistic models is a core problem in machine learning, which presents significant challenges due to the curse of dimensionality. This paper proposes a joint dimensionality reduction and non-parametric density estimation framework, using a novel estimator that can explicitly capture the underlying distribution of appropriate reduced-dimension representations of the input data. The idea is to jointly design a nonlinear dimensionality reducing auto-encoder to model the training data in terms of a parsimonious set of latent random variables, and learn a canonical low-rank tensor model of the joint distribution of the latent variables in the Fourier domain. The proposed latent density model is non-parametric and universal, as opposed to the predefined prior that is assumed in variational auto-encoders. Joint optimization of the auto-encoder and the latent density estimator is pursued via a formulation which learns both by minimizing a combination of the negative log-likelihood in the latent domain and the auto-encoder reconstruction loss. We demonstrate that the proposed model achieves very promising results on toy, tabular, and image datasets on regression tasks, sampling, and anomaly detection.