Free Lunch for Optimisation under the Universal Distribution
This addresses a foundational problem in optimization theory by offering a theoretical framework that could impact algorithm design across machine learning and AI.
The paper challenges the No Free Lunch theorems by proposing a universal prior that allows for a free lunch in function optimization, where no algorithm is universally superior, and provides upper and lower bounds on its extent.
Function optimisation is a major challenge in computer science. The No Free Lunch theorems state that if all functions with the same histogram are assumed to be equally probable then no algorithm outperforms any other in expectation. We argue against the uniform assumption and suggest a universal prior exists for which there is a free lunch, but where no particular class of functions is favoured over another. We also prove upper and lower bounds on the size of the free lunch.