Accelerating likelihood optimization for ICA on real signals
This addresses a performance bottleneck for ICA practitioners dealing with real-world data, offering an incremental improvement over existing methods.
The paper tackled the slow convergence of quasi-Newton methods for maximum likelihood ICA on real signals, showing that the Picard algorithm overcomes this issue for both constrained and unconstrained problems.
We study optimization methods for solving the maximum likelihood formulation of independent component analysis (ICA). We consider both the the problem constrained to white signals and the unconstrained problem. The Hessian of the objective function is costly to compute, which renders Newton's method impractical for large data sets. Many algorithms proposed in the literature can be rewritten as quasi-Newton methods, for which the Hessian approximation is cheap to compute. These algorithms are very fast on simulated data where the linear mixture assumption really holds. However, on real signals, we observe that their rate of convergence can be severely impaired. In this paper, we investigate the origins of this behavior, and show that the recently proposed Preconditioned ICA for Real Data (Picard) algorithm overcomes this issue on both constrained and unconstrained problems.