Cauchy noise loss for stochastic optimization of random matrix models via free deterministic equivalents
This addresses parameter estimation challenges in random matrix models for researchers in statistics and machine learning, though it appears incremental as it builds on existing free probability tools.
The authors tackled parameter estimation for random matrix models when only one sample matrix is available by introducing a method based on spectral distributions rather than traditional likelihoods, using Cauchy noise and free deterministic equivalents to approximate distributions; experimentally, their method recovered the rank of signal parts even when the true rank was not small.
For random matrix models, the parameter estimation based on the traditional likelihood functions is not straightforward in particular when we have only one sample matrix. We introduce a new parameter optimization method for random matrix models which works even in such a case. The method is based on the spectral distribution instead of the traditional likelihood. In the method, the Cauchy noise has an essential role because the free deterministic equivalent, which is a tool in free probability theory, allows us to approximate the spectral distribution perturbed by Cauchy noises by a smooth and accessible density function. Moreover, we study an asymptotic property of determination gap, which has a similar role as generalization gap. Besides, we propose a new dimensionality recovery method for the signal-plus-noise model, and experimentally demonstrate that it recovers the rank of the signal part even if the true rank is not small. It is a simultaneous rank selection and parameter estimation procedure.