Learning to Guide Local Search for MPE Inference in Probabilistic Graphical Models
This work addresses a fundamental inference problem in domains such as diagnosis and planning, offering an incremental improvement for scenarios where graphical models are fixed and queries vary.
The paper tackles the problem of Most Probable Explanation (MPE) inference in Probabilistic Graphical Models, which is computationally challenging in repeated-query settings, by proposing a neural amortization framework that improves local search algorithms, resulting in consistent empirical improvements over existing methods like SLS and GLS+ on high-treewidth benchmarks.
Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs) is a fundamental yet computationally challenging problem arising in domains such as diagnosis, planning, and structured prediction. In many practical settings, the graphical model remains fixed while inference must be performed repeatedly for varying evidence patterns. Stochastic Local Search (SLS) algorithms scale to large models but rely on myopic best-improvement rule that prioritizes immediate likelihood gains and often stagnate in poor local optima. Heuristics such as Guided Local Search (GLS+) partially alleviate this limitation by modifying the search landscape, but their guidance cannot be reused effectively across multiple inference queries on the same model. We propose a neural amortization framework for improving local search in this repeated-query regime. Exploiting the fixed graph structure, we train an attention-based network to score local moves by predicting their ability to reduce Hamming distance to a near-optimal solution. Our approach integrates seamlessly with existing local search procedures, using this signal to balance short-term likelihood gains with long-term promise during neighbor selection. We provide theoretical intuition linking distance-reducing move selection to improved convergence behavior, and empirically demonstrate consistent improvements over SLS and GLS+ on challenging high-treewidth benchmarks in the amortized inference setting.