A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods
For practitioners of Bayesian inference, it provides a principled, scalable variational method with theoretical guarantees for a broad class of models, addressing a gap in existing black-box approaches.
This paper proposes a variational inference framework using tangent approximation for strongly super-Gaussian likelihoods, achieving algorithmic convergence guarantees and near-minimax optimal risk bounds. It outperforms state-of-the-art methods in scalability and accuracy on simulated and real data.
Variational inference, as an alternative to Markov chain Monte Carlo sampling, has played a transformative role in enabling scalable computation for complex Bayesian models. Nevertheless, existing approaches often depend on either rigid model-specific formulations or stochastic black-box optimization routines. Tangent approximation is a principled class of structured variational methods that exploits the geometry of the underlying probability model. However, its utility has largely been confined to logistic regression and related modeling regimes. In this article, we propose a novel variational framework based on tangent transformation for a broad class of probability models characterized by strongly super-Gaussian likelihoods. Our method leverages convex duality to construct tangent minorants of the log-likelihood, thereby inducing conjugacy with Gaussian priors over model parameters in an otherwise intractable setup. Under mild assumptions on the data-generating mechanism, we establish algorithmic convergence guarantees, a contribution that stands in contrast to the limited theoretical assurances typically available for black-box variational methods. Additionally, we derive near-minimax optimal bounds for the variational risk. Superior performance of our proposed methodology is illustrated on simulated and real-data scenarios that challenge state-of-the-art variational algorithms in terms of scalability and their ability to consistently capture complex underlying data structure.