The Einstein Test: Towards a Practical Test of a Machine's Ability to Exhibit Superintelligence
This work addresses the challenge of testing machine superintelligence for researchers and AI developers, but it is incremental as it builds on existing debates about AI capabilities without introducing new computational methods.
The paper tackles the problem of evaluating whether AI can generate creative and disruptive insights like Einstein's theory of relativity, proposing the Einstein test as a practical method to assess if AI can independently reproduce such insights from historical data.
Creative and disruptive insights (CDIs), such as the development of the theory of relativity, have punctuated human history, marking pivotal shifts in our intellectual trajectory. Recent advancements in artificial intelligence (AI) have sparked debates over whether state of the art models possess the capacity to generate CDIs. We argue that the ability to create CDIs should be regarded as a significant feature of machine superintelligence (SI).To this end, we propose a practical test to evaluate whether an approach to AI targeting SI can yield novel insights of this kind. We propose the Einstein test: given the data available prior to the emergence of a known CDI, can an AI independently reproduce that insight (or one that is formally equivalent)? By achieving such a milestone, a machine can be considered to at least match humanity's past top intellectual achievements, and therefore to have the potential to surpass them.