Optimal scaling of random-walk Metropolis algorithms using Bayesian large-sample asymptotics
This work addresses a practical issue for users of MCMC methods in statistics and machine learning by providing more robust tuning guidelines under realistic assumptions, though it is incremental as it builds on existing scaling theories.
The paper tackles the problem of tuning random-walk Metropolis algorithms under restrictive assumptions, using Bayesian large-sample asymptotics to prove weak convergence under realistic conditions and propose novel parameter-dimension-dependent tuning guidelines. The results show consistency with previous rules for near-product-form targets and highlight the need to account for correlation structure to avoid performance deterioration, with an asymptotically exact approximation usable from the first run.
High-dimensional limit theorems have been shown useful to derive tuning rules for finding the optimal scaling in random-walk Metropolis algorithms. The assumptions under which weak convergence results are proved are however restrictive: the target density is typically assumed to be of a product form. Users may thus doubt the validity of such tuning rules in practical applications. In this paper, we shed some light on optimal-scaling problems from a different perspective, namely a large-sample one. This allows to prove weak convergence results under realistic assumptions and to propose novel parameter-dimension-dependent tuning guidelines. The proposed guidelines are consistent with previous ones when the target density is close to having a product form, and the results highlight that the correlation structure has to be accounted for to avoid performance deterioration if that is not the case, while justifying the use of a natural (asymptotically exact) approximation to the correlation matrix that can be employed for the very first algorithm run.