43.0NAMay 18
Convergence for adaptive resampling of random Fourier featuresXin Huang, Aku Kammonen, Anamika Pandey et al.
The machine learning random Fourier feature method for data in high dimension is computationally and theoretically attractive since the optimization is based on a convex standard least squares problem and independent sampling of Fourier frequencies. The challenge is to sample the Fourier frequencies well. This work proves convergence of a data adaptive method based on resampling the frequencies asymptotically optimally, as the number of nodes and amount of data tend to infinity. Numerical results based on resampling and adaptive random walk steps together with approximations of the least squares problem by conjugate gradient iterations confirm the analysis for regression and classification problems.
LGFeb 1, 2024
Comparing Spectral Bias and Robustness For Two-Layer Neural Networks: SGD vs Adaptive Random Fourier FeaturesAku Kammonen, Lisi Liang, Anamika Pandey et al.
We present experimental results highlighting two key differences resulting from the choice of training algorithm for two-layer neural networks. The spectral bias of neural networks is well known, while the spectral bias dependence on the choice of training algorithm is less studied. Our experiments demonstrate that an adaptive random Fourier features algorithm (ARFF) can yield a spectral bias closer to zero compared to the stochastic gradient descent optimizer (SGD). Additionally, we train two identically structured classifiers, employing SGD and ARFF, to the same accuracy levels and empirically assess their robustness against adversarial noise attacks.
LGJul 21, 2025
An Adaptive Random Fourier Features approach Applied to Learning Stochastic Differential EquationsOwen Douglas, Aku Kammonen, Anamika Pandey et al.
This work proposes a training algorithm based on adaptive random Fourier features (ARFF) with Metropolis sampling and resampling \cite{kammonen2024adaptiverandomfourierfeatures} for learning drift and diffusion components of stochastic differential equations from snapshot data. Specifically, this study considers Itô diffusion processes and a likelihood-based loss function derived from the Euler-Maruyama integration introduced in \cite{Dietrich2023} and \cite{dridi2021learningstochasticdynamicalsystems}. This work evaluates the proposed method against benchmark problems presented in \cite{Dietrich2023}, including polynomial examples, underdamped Langevin dynamics, a stochastic susceptible-infected-recovered model, and a stochastic wave equation. Across all cases, the ARFF-based approach matches or surpasses the performance of conventional Adam-based optimization in both loss minimization and convergence speed. These results highlight the potential of ARFF as a compelling alternative for data-driven modeling of stochastic dynamics.
NAFeb 20, 2012
Meshfree method for fluctuating hydrodynamicsAnamika Pandey, Axel Klar, Sudarshan Tiwari
In the current study a meshfree Lagrangian particle method for the Landau-Lifshitz Navier-Stokes (LLNS) equations is developed. The LLNS equations incorporate thermal fluctuation into macroscopic hydrodynamics by the addition of white noise fluxes whose magnitudes are set by a fluctuation-dissipation theorem. The study focuses on capturing the correct variance and correlations computed at equilibrium flows, which are compared with available theoretical values. Moreover, a numerical test for the random walk of standing shock wave has been considered for capturing the shock location.