Position-wise optimizer: A nature-inspired optimization algorithm
This work addresses optimization in neural networks for researchers, but it appears incremental as it builds on existing ideas of plasticity without clear broad impact.
The authors tackled the problem of optimizing artificial neural networks by introducing a nature-inspired algorithm based on biological neural plasticity, achieving results comparable to gradient descent on three datasets.
The human nervous system utilizes synaptic plasticity to solve optimization problems. Previous studies have tried to add the plasticity factor to the training process of artificial neural networks, but most of those models require complex external control over the network or complex novel rules. In this manuscript, a novel nature-inspired optimization algorithm is introduced that imitates biological neural plasticity. Furthermore, the model is tested on three datasets and the results are compared with gradient descent optimization.