Cross-Language Assessment of Mathematical Capability of ChatGPT
This addresses the need for assessing AI capabilities in underrepresented languages, though it is incremental as it extends existing evaluation methods to new contexts.
The paper evaluated ChatGPT's mathematical problem-solving accuracy across Hindi, Gujarati, and Marathi, finding that chain-of-thought prompting improved performance but with varying effectiveness compared to English.
This paper presents an evaluation of the mathematical capability of ChatGPT across diverse languages like Hindi, Gujarati, and Marathi. ChatGPT, based on GPT-3.5 by OpenAI, has garnered significant attention for its natural language understanding and generation abilities. However, its performance in solving mathematical problems across multiple natural languages remains a comparatively unexplored area, especially in regional Indian languages. In this paper, we explore those capabilities as well as using chain-of-thought prompting to figure out if it increases the accuracy of responses as much as it does in the English language and provide insights into the current limitations.