An Informal Introduction to Multiplet Neural Networks
This work proposes a novel neuron architecture for improving function approximation in neural networks, though it appears incremental as it builds on generalized means.
The authors tackled the problem of enhancing neural network expressiveness by replacing the dot product in an artificial neuron with a weighted Lehmer mean and using a multiplet of neurons with shared averaging weights, which enabled the network to emulate the XOR problem in two layers and perform multiplication and division.
In the artificial neuron, I replace the dot product with the weighted Lehmer mean, which may emulate different cases of a generalized mean. The single neuron instance is replaced by a multiplet of neurons which have the same averaging weights. A group of outputs feed forward, in lieu of the single scalar. The generalization parameter is typically set to a different value for each neuron in the multiplet. I further extend the concept to a multiplet taken from the Gini mean. Derivatives with respect to the weight parameters and with respect to the two generalization parameters are given. Some properties of the network are investigated, showing the capacity to emulate the classical exclusive-or problem organically in two layers and perform some multiplication and division. The network can instantiate truncated power series and variants, which can be used to approximate different functions, provided that parameters are constrained. Moreover, a mean case slope score is derived that can facilitate a learning-rate novelty based on homogeneity of the selected elements. The multiplet neuron equation provides a way to segment regularization timeframes and approaches.