Univariate Radial Basis Function Layers: Brain-inspired Deep Neural Layers for Low-Dimensional Inputs
This addresses the need for specialized architectures in low-dimensional real-world problems, offering an incremental improvement over existing methods.
The paper tackles the problem of deep neural networks for low-dimensional inputs by proposing a novel Univariate Radial Basis Function (U-RBF) layer, showing it outperforms standard Multi-Layer Perceptrons in complex low-dimensional function regressions and reinforcement learning tasks.
Deep Neural Networks (DNNs) became the standard tool for function approximation with most of the introduced architectures being developed for high-dimensional input data. However, many real-world problems have low-dimensional inputs for which standard Multi-Layer Perceptrons (MLPs) are the default choice. An investigation into specialized architectures is missing. We propose a novel DNN layer called Univariate Radial Basis Function (U-RBF) layer as an alternative. Similar to sensory neurons in the brain, the U-RBF layer processes each individual input dimension with a population of neurons whose activations depend on different preferred input values. We verify its effectiveness compared to MLPs in low-dimensional function regressions and reinforcement learning tasks. The results show that the U-RBF is especially advantageous when the target function becomes complex and difficult to approximate.