q-Neurons: Neuron Activations based on Stochastic Jackson's Derivative Operators
This work addresses the need for better activation functions in neural networks, offering a scalable and easy-to-implement solution, though it appears incremental as it builds on existing derivative-based methods.
The authors tackled the problem of improving neural network performance by proposing q-neurons, a new type of stochastic neuron based on Jackson's q-derivatives with stochastic parameters, and demonstrated consistently improved performances over state-of-the-art standard activation functions on training and testing loss functions.
We propose a new generic type of stochastic neurons, called $q$-neurons, that considers activation functions based on Jackson's $q$-derivatives with stochastic parameters $q$. Our generalization of neural network architectures with $q$-neurons is shown to be both scalable and very easy to implement. We demonstrate experimentally consistently improved performances over state-of-the-art standard activation functions, both on training and testing loss functions.