Constructing the Matrix Multilayer Perceptron and its Application to the VAE
This work addresses the problem of learning structured matrix parameters in machine learning, particularly for VAEs, but is incremental as it builds on existing MLP and VAE frameworks.
The authors introduced a matrix multilayer perceptron (MLP) variant specialized for learning symmetric positive definite matrices and applied it to extend variational autoencoders (VAEs) to handle dense covariance matrices, demonstrating results on synthetic and real data.
Like most learning algorithms, the multilayer perceptrons (MLP) is designed to learn a vector of parameters from data. However, in certain scenarios we are interested in learning structured parameters (predictions) in the form of symmetric positive definite matrices. Here, we introduce a variant of the MLP, referred to as the matrix MLP, that is specialized at learning symmetric positive definite matrices. We also present an application of the model within the context of the variational autoencoder (VAE). Our formulation of the VAE extends the vanilla formulation to the cases where the recognition and the generative networks can be from the parametric family of distributions with dense covariance matrices. Two specific examples are discussed in more detail: the dense covariance Gaussian and its generalization, the power exponential distribution. Our new developments are illustrated using both synthetic and real data.