Comparative Analysis of ChatGPT, GPT-4, and Microsoft Bing Chatbots for GRE Test
This study assesses AI chatbots for standardized test performance, which is incremental as it applies existing methods to a new educational domain.
The paper analyzed the performance of Bing, ChatGPT, and GPT-4 on GRE test questions, finding that GPT-4 was the most proficient, especially in complex language and image-based tasks, with results indicating these chatbots can achieve high scores.
This research paper presents an analysis of how well three artificial intelligence chatbots: Bing, ChatGPT, and GPT-4, perform when answering questions from standardized tests. The Graduate Record Examination is used in this paper as a case study. A total of 137 questions with different forms of quantitative reasoning and 157 questions with verbal categories were used to assess their capabilities. This paper presents the performance of each chatbot across various skills and styles tested in the exam. The proficiency of these chatbots in addressing image-based questions is also explored, and the uncertainty level of each chatbot is illustrated. The results show varying degrees of success across the chatbots, where GPT-4 served as the most proficient, especially in complex language understanding tasks and image-based questions. Results highlight the ability of these chatbots to pass the GRE with a high score, which encourages the use of these chatbots in test preparation. The results also show how important it is to ensure that, if the test is administered online, as it was during COVID, the test taker is segregated from these resources for a fair competition on higher education opportunities.