Double-Bayesian Learning
This work addresses foundational issues in ML decision-making, but it appears incremental as it builds on existing Bayesian concepts without clear empirical validation.
The paper tackles the problem of decision-making in machine learning by proposing that decisions are composed of two Bayesian processes, leading to intrinsic uncertainty and explainability. It shows that this double-Bayesian framework implies solutions described by the golden ratio and suggests using learning rates and momentum weights similar to those in neural network training.
Contemporary machine learning methods will try to approach the Bayes error, as it is the lowest possible error any model can achieve. This paper postulates that any decision is composed of not one but two Bayesian decisions and that decision-making is, therefore, a double-Bayesian process. The paper shows how this duality implies intrinsic uncertainty in decisions and how it incorporates explainability. The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points. Furthermore, following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem. The double-Bayesian framework suggests using a learning rate and momentum weight with values similar to those used in the literature to train neural networks with stochastic gradient descent.