A Bayesian Approach to Invariant Deep Neural Networks
This addresses the challenge of improving model robustness to invariances for machine learning practitioners, but it is incremental as it builds on existing Bayesian and invariance methods.
The authors tackled the problem of learning invariances in deep neural networks without data augmentation by proposing a Bayesian architecture that infers weight-sharing schemes, resulting in outperforming non-invariant models on datasets with specific invariances.
We propose a novel Bayesian neural network architecture that can learn invariances from data alone by inferring a posterior distribution over different weight-sharing schemes. We show that our model outperforms other non-invariant architectures, when trained on datasets that contain specific invariances. The same holds true when no data augmentation is performed.