FutureGen: A RAG-based Approach to Generate the Future Work of Scientific Article
This addresses the need for automated generation of research directions in scientific articles, which is incremental as it builds on existing RAG and LLM techniques.
The study tackled the problem of generating future work suggestions for scientific articles by using a Retrieval-Augmented Generation (RAG) approach with LLMs, and found that GPT-4o mini combined with an LLM feedback mechanism outperformed other methods in evaluations.
The Future Work section of a scientific article outlines potential research directions by identifying gaps and limitations of a current study. This section serves as a valuable resource for early-career researchers seeking unexplored areas and experienced researchers looking for new projects or collaborations. In this study, we generate future work suggestions from a scientific article. To enrich the generation process with broader insights and reduce the chance of missing important research directions, we use context from related papers using RAG. We experimented with various Large Language Models (LLMs) integrated into Retrieval-Augmented Generation (RAG). We incorporate an LLM feedback mechanism to enhance the quality of the generated content and introduce an LLM-as-a-judge framework for robust evaluation, assessing key aspects such as novelty, hallucination, and feasibility. Our results demonstrate that the RAG-based approach using GPT-4o mini, combined with an LLM feedback mechanism, outperforms other methods based on both qualitative and quantitative evaluations. Moreover, we conduct a human evaluation to assess the LLM as an extractor, generator, and feedback provider.