Evaluating AI Evaluation: Perils and Prospects
This addresses the critical problem of ensuring safe and reliable AI development for researchers and policymakers, but it is incremental as it builds on existing ideas without introducing a new method.
The paper argues that current evaluation methods for AI systems are inadequate and increase risks, proposing a reformation inspired by cognitive sciences to assess general intelligence and improve safety.
As AI systems appear to exhibit ever-increasing capability and generality, assessing their true potential and safety becomes paramount. This paper contends that the prevalent evaluation methods for these systems are fundamentally inadequate, heightening the risks and potential hazards associated with AI. I argue that a reformation is required in the way we evaluate AI systems and that we should look towards cognitive sciences for inspiration in our approaches, which have a longstanding tradition of assessing general intelligence across diverse species. We will identify some of the difficulties that need to be overcome when applying cognitively-inspired approaches to general-purpose AI systems and also analyse the emerging area of "Evals". The paper concludes by identifying promising research pathways that could refine AI evaluation, advancing it towards a rigorous scientific domain that contributes to the development of safe AI systems.