Variational Bayesian Pseudo-Coreset
This work addresses memory inefficiency and sub-optimal results in Bayesian Pseudo-Coresets for deep learning practitioners, representing an incremental improvement.
The paper tackles the computational challenges of Bayesian Neural Networks with large datasets by proposing Variational Bayesian Pseudo-Coreset (VBPC), which uses variational inference to approximate the posterior distribution, reducing memory usage and computational costs while improving performance on benchmark datasets.
The success of deep learning requires large datasets and extensive training, which can create significant computational challenges. To address these challenges, pseudo-coresets, small learnable datasets that mimic the entire data, have been proposed. Bayesian Neural Networks, which offer predictive uncertainty and probabilistic interpretation for deep neural networks, also face issues with large-scale datasets due to their high-dimensional parameter space. Prior works on Bayesian Pseudo-Coresets (BPC) attempt to reduce the computational load for computing weight posterior distribution by a small number of pseudo-coresets but suffer from memory inefficiency during BPC training and sub-optimal results. To overcome these limitations, we propose Variational Bayesian Pseudo-Coreset (VBPC), a novel approach that utilizes variational inference to efficiently approximate the posterior distribution, reducing memory usage and computational costs while improving performance across benchmark datasets.