A Multiplicative Neural Network Architecture: Locality and Regularity of Approximation
This work addresses the challenge of improving approximation properties in neural networks for researchers in mathematical machine learning, though it appears incremental as it builds on existing architectures with a focus on analytical properties.
The paper tackles the problem of neural network approximation by introducing a multiplicative architecture that uses multiplicative interactions as the fundamental representation, establishing a universal approximation theorem and showing through experiments that it yields error structures more aligned with regions of reduced regularity and stable convergence in regularity-sensitive metrics.
We introduce a multiplicative neural network architecture in which multiplicative interactions constitute the fundamental representation, rather than appearing as auxiliary components within an additive model. We establish a universal approximation theorem for this architecture and analyze its approximation properties in terms of locality and regularity in Bessel potential spaces. To complement the theoretical results, we conduct numerical experiments on representative targets exhibiting sharp transition layers or pointwise loss of higher-order regularity. The experiments focus on the spatial structure of approximation errors and on regularity-sensitive quantities, in particular, the convergence of Zygmund-type seminorms. The results show that the proposed multiplicative architecture yields residual error structures that are more tightly aligned with regions of reduced regularity and exhibit more stable convergence in regularity-sensitive metrics. These results demonstrate that adopting a multiplicative representation format has concrete implications for the localization and regularity behavior of neural network approximations, providing a direct connection between architectural design and analytical properties of the approximating functions.