Frequency-adaptive tensor neural networks for high-dimensional multi-scale problems
This work addresses a bottleneck in solving complex multi-scale problems for researchers and practitioners in computational science and machine learning, though it is incremental in enhancing existing TNN methods.
The paper tackled the limitation of tensor neural networks (TNNs) in capturing high-frequency features due to the Frequency Principle, and proposed a frequency-adaptive TNNs algorithm that significantly improves performance on high-dimensional multi-scale problems, as validated by extensive numerical experiments.
Tensor neural networks (TNNs) have demonstrated their superiority in solving high-dimensional problems. However, similar to conventional neural networks, TNNs are also influenced by the Frequency Principle, which limits their ability to accurately capture high-frequency features of the solution. In this work, we analyze the training dynamics of TNNs by Fourier analysis and enhance their expressivity for high-dimensional multi-scale problems by incorporating random Fourier features. Leveraging the inherent tensor structure of TNNs, we further propose a novel approach to extract frequency features of high-dimensional functions by performing the Discrete Fourier Transform to one-dimensional component functions. This strategy effectively mitigates the curse of dimensionality. Building on this idea, we propose a frequency-adaptive TNNs algorithm, which significantly improves the ability of TNNs in solving complex multi-scale problems. Extensive numerical experiments are performed to validate the effectiveness and robustness of the proposed frequency-adaptive TNNs algorithm.