On limitations of learning algorithms in competitive environments
This work identifies fundamental, theoretical limitations for general learning algorithms in competitive settings, which is significant for researchers developing AI agents for adversarial scenarios.
This paper explores the fundamental limitations of general learning algorithms operating in competitive, adversarial environments. It demonstrates that these algorithms face constraints on their knowledge analogous to those found in Gödel's incompleteness theorems and Turing's undecidability results, linking these limitations to the inherent intransitivity often present in such environments.
We discuss conceptual limitations of generic learning algorithms pursuing adversarial goals in competitive environments, and prove that they are subject to limitations that are analogous to the constraints on knowledge imposed by the famous theorems of Gödel and Turing. These limitations are shown to be related to intransitivity, which is commonly present in competitive environments.