Optimisation Is Not What You Need
This work challenges the foundational approach to AI by identifying critical limitations in optimization for achieving artificial general intelligence, which could redirect research efforts in the field.
The paper argues that optimization methods in AI, despite achieving results like passing the Turing Test, have inherent flaws such as catastrophic forgetting and overfitting that prevent true artificial cognition, and it empirically shows that world-modeling methods avoid these issues.
The Artificial Intelligence field has focused on developing optimisation methods to solve multiple problems, specifically problems that we thought to be only solvable through cognition. The obtained results have been outstanding, being able to even surpass the Turing Test. However, we have found that these optimisation methods share some fundamental flaws that impede them to become a true artificial cognition. Specifically, the field have identified catastrophic forgetting as a fundamental problem to develop such cognition. This paper formally proves that this problem is inherent to optimisation methods, and as such it will always limit approaches that try to solve the Artificial General Intelligence problem as an optimisation problem. Additionally, it addresses the problem of overfitting and discuss about other smaller problems that optimisation methods pose. Finally, it empirically shows how world-modelling methods avoid suffering from either problem. As a conclusion, the field of Artificial Intelligence needs to look outside the machine learning field to find methods capable of developing an artificial cognition.