Solomonoff-Inspired Hypothesis Ranking with LLMs for Prediction Under Uncertainty
This work addresses the problem of balancing accuracy and simplicity in multi-hypothesis reasoning under uncertainty for AI tasks, representing an incremental improvement by applying algorithmic information-theoretic priors to existing methods.
The paper tackled the challenge of reasoning under uncertainty in AI by proposing a Solomonoff-inspired method that weights LLM-generated hypotheses based on simplicity and predictive fit, applied to Mini-ARC benchmark tasks, resulting in conservative, uncertainty-aware predictions that spread probability more evenly across hypotheses compared to Bayesian Model Averaging.
Reasoning under uncertainty is a key challenge in AI, especially for real-world tasks, where problems with sparse data demands systematic generalisation. Existing approaches struggle to balance accuracy and simplicity when evaluating multiple candidate solutions. We propose a Solomonoff-inspired method that weights LLM-generated hypotheses by simplicity and predictive fit. Applied to benchmark (Mini-ARC) tasks, our method produces Solomonoff-weighted mixtures for per-cell predictions, yielding conservative, uncertainty-aware outputs even when hypotheses are noisy or partially incorrect. Compared to Bayesian Model Averaging (BMA), Solomonoff scoring spreads probability more evenly across competing hypotheses, while BMA concentrates weight on the most likely but potentially flawed candidates. Across tasks, this highlights the value of algorithmic information-theoretic priors for interpretable, reliable multi-hypothesis reasoning under uncertainty.