Automatic Item Generation of Figural Analogy Problems: A Review and Outlook
This work addresses the need for systematic item generation in cognitive science and AI, but it is incremental as it reviews and compares existing methods without introducing new techniques.
The paper reviews algorithmic approaches for automatically generating figural analogy problems, used in human intelligence tests and data-driven AI models, analyzing principles and suggesting future research directions.
Figural analogy problems have long been a widely used format in human intelligence tests. In the past four decades, more and more research has investigated automatic item generation for figural analogy problems, i.e., algorithmic approaches for systematically and automatically creating such problems. In cognitive science and psychometrics, this research can deepen our understandings of human analogical ability and psychometric properties of figural analogies. With the recent development of data-driven AI models for reasoning about figural analogies, the territory of automatic item generation of figural analogies has further expanded. This expansion brings new challenges as well as opportunities, which demand reflection on previous item generation research and planning future studies. This paper reviews the important works of automatic item generation of figural analogies for both human intelligence tests and data-driven AI models. From an interdisciplinary perspective, the principles and technical details of these works are analyzed and compared, and desiderata for future research are suggested.