Active Continual Learning with Metaplastic Binary Bayesian Neural Networks
For edge systems with tight compute budgets, BiMU provides a practical solution to maintain plasticity and uncertainty estimation in long non-stationary streams, reducing label and update costs.
BiMU, a method for active continual learning with binary Bayesian neural networks, prevents posterior saturation and sustains epistemic uncertainty, enabling buffer-free active querying. It achieves up to 32x label/update savings at matched accuracy on OpenLORIS-Object under class imbalance.
Always-on edge systems must keep learning as conditions change under tight compute budgets and must detect unreliable predictions. Bayesian binary neural networks are attractive in this setting, but mean-field Bernoulli posteriors can saturate on long non-stationary streams, wiping out epistemic uncertainty and freezing plasticity. We propose BiMU, derived from a bounded-memory variational objective that balances stability, plasticity, and forgetting. BiMU combines a data term with controlled relaxation toward the prior and an uncertainty-dependent step size that prevents saturation and sustains informative uncertainty. This non-degenerate posterior enables fully online, buffer-free active querying via Monte Carlo disagreement, reducing label queries and backpropagation updates under imbalance. BiMU sustains learning and strong OOD detection on 1000-tasks Permuted-MNIST, and on OpenLORIS-Object achieves up to 32$\times$ label/update savings at matched accuracy under class imbalance and feature compression.