Do androids dream of fictional references? A bibliographic dialogue with ChatGPT3.5
This highlights a critical reliability problem for researchers using AI-generated references, though it is incremental as it builds on known issues with AI hallucinations.
The study analyzed bibliographic references generated by ChatGPT3.5 across six themes in French and English, finding high percentages of fictitious references, with some improvement in English over time but persistent issues in French.
This article focuses on bibliographic references generated by the ChatGPT3.5 tool. Using this tool based on the trained GPT generation model ChatGPT3.5, developed by the company OpenAI, we explored six different themes and analyzed a sample of references generated by the model, in French and English. The results revealed high percentages of fictitious references in several fields, underlining the importance of carefully checking these references before using them in research work. An improvement in results was nevertheless noted between May and July with regard to English references for themes on which ChatGPR3.5 has been particularly trained, but the situation remains unsatisfactory in French, for example. It should also be pointed out that much of the text in this article was generated by ChatGPT in a joint effort with the human author.