AI Insights: A Case Study on Utilizing ChatGPT Intelligence for Research Paper Analysis
This work addresses the challenge of efficient scientific literature review for researchers, but it is incremental as it applies existing AI models to a specific domain.
This paper tackles the problem of automating research paper analysis for literature surveys by evaluating ChatGPT versions 3.5 and 4 on breast cancer treatment papers, finding that GPT-4 achieves 77.3% accuracy in category identification and 50% in scope identification.
This paper discusses the effectiveness of leveraging Chatbot: Generative Pre-trained Transformer (ChatGPT) versions 3.5 and 4 for analyzing research papers for effective writing of scientific literature surveys. The study selected the \textit{Application of Artificial Intelligence in Breast Cancer Treatment} as the research topic. Research papers related to this topic were collected from three major publication databases Google Scholar, Pubmed, and Scopus. ChatGPT models were used to identify the category, scope, and relevant information from the research papers for automatic identification of relevant papers related to Breast Cancer Treatment (BCT), organization of papers according to scope, and identification of key information for survey paper writing. Evaluations performed using ground truth data annotated using subject experts reveal, that GPT-4 achieves 77.3\% accuracy in identifying the research paper categories and 50\% of the papers were correctly identified by GPT-4 for their scopes. Further, the results demonstrate that GPT-4 can generate reasons for its decisions with an average of 27\% new words, and 67\% of the reasons given by the model were completely agreeable to the subject experts.