Bayesian Neural Networks: A Min-Max Game Framework
This work addresses robustness issues in Bayesian neural networks for machine learning practitioners, but it appears incremental as it builds on existing minimax and BNN concepts.
The paper tackles the problem of robustness in Bayesian neural networks by formulating a two-player min-max game between deterministic and stochastic networks, revealing connections to BNNs and testing on simple datasets with noise perturbations.
In deep learning, Bayesian neural networks (BNN) provide the role of robustness analysis, and the minimax method is used to be a conservative choice in the traditional Bayesian field. In this paper, we study a conservative BNN with the minimax method and formulate a two-player game between a deterministic neural network $f$ and a sampling stochastic neural network $f + r*ξ$. From this perspective, we understand the closed-loop neural networks with the minimax loss and reveal their connection to the BNN. We test the models on simple data sets, study their robustness under noise perturbation, and report some issues for searching $r$.