Unbiased Weight Maximization
This addresses the structural credit assignment problem in biologically inspired AI, offering a scalable solution for training networks of stochastic units, though it is incremental as it builds on prior Weight Maximization.
The paper tackles the slow learning and poor scaling of biologically plausible neural network training via REINFORCE by proposing Unbiased Weight Maximization, which replaces the global reward signal with an unbiased norm-based rule, resulting in increased learning speed and improved asymptotic performance.
A biologically plausible method for training an Artificial Neural Network (ANN) involves treating each unit as a stochastic Reinforcement Learning (RL) agent, thereby considering the network as a team of agents. Consequently, all units can learn via REINFORCE, a local learning rule modulated by a global reward signal, which aligns more closely with biologically observed forms of synaptic plasticity. Nevertheless, this learning method is often slow and scales poorly with network size due to inefficient structural credit assignment, since a single reward signal is broadcast to all units without considering individual contributions. Weight Maximization, a proposed solution, replaces a unit's reward signal with the norm of its outgoing weight, thereby allowing each hidden unit to maximize the norm of the outgoing weight instead of the global reward signal. In this research report, we analyze the theoretical properties of Weight Maximization and propose a variant, Unbiased Weight Maximization. This new approach provides an unbiased learning rule that increases learning speed and improves asymptotic performance. Notably, to our knowledge, this is the first learning rule for a network of Bernoulli-logistic units that is unbiased and scales well with the number of network's units in terms of learning speed.