Variational Autoencoder Kernel Interpretation and Selection for Classification
This work addresses model efficiency for resource-constrained devices, but it is incremental as it builds on existing variational autoencoder and feature selection techniques.
The paper tackled the problem of selecting relevant kernels in variational autoencoders for classification, resulting in a method that reduces model parameters by discarding uninformative kernels based on distribution similarity, with confirmation via Kullback-Leibler divergence.
This work proposed kernel selection approaches for probabilistic classifiers based on features produced by the convolutional encoder of a variational autoencoder. Particularly, the developed methodologies allow the selection of the most relevant subset of latent variables. In the proposed implementation, each latent variable was sampled from the distribution associated with a single kernel of the last encoder's convolution layer, as an individual distribution was created for each kernel. Therefore, choosing relevant features on the sampled latent variables makes it possible to perform kernel selection, filtering the uninformative features and kernels. Such leads to a reduction in the number of the model's parameters. Both wrapper and filter methods were evaluated for feature selection. The second was of particular relevance as it is based only on the distributions of the kernels. It was assessed by measuring the Kullback-Leibler divergence between all distributions, hypothesizing that the kernels whose distributions are more similar can be discarded. This hypothesis was confirmed since it was observed that the most similar kernels do not convey relevant information and can be removed. As a result, the proposed methodology is suitable for developing applications for resource-constrained devices.