SciMON: Scientific Inspiration Machines Optimized for Novelty
This work addresses the need for more expressive and novel hypothesis generation in scientific research, representing an incremental step in developing language models for idea generation.
The paper tackled the problem of generating novel scientific ideas from literature, moving beyond binary link prediction to produce natural language suggestions, and found that their SciMON framework partially improved novelty compared to GPT-4's low novelty outputs.
We explore and enhance the ability of neural language models to generate novel scientific directions grounded in literature. Work on literature-based hypothesis generation has traditionally focused on binary link prediction--severely limiting the expressivity of hypotheses. This line of work also does not focus on optimizing novelty. We take a dramatic departure with a novel setting in which models use as input background contexts (e.g., problems, experimental settings, goals), and output natural language ideas grounded in literature. We present SciMON, a modeling framework that uses retrieval of "inspirations" from past scientific papers, and explicitly optimizes for novelty by iteratively comparing to prior papers and updating idea suggestions until sufficient novelty is achieved. Comprehensive evaluations reveal that GPT-4 tends to generate ideas with overall low technical depth and novelty, while our methods partially mitigate this issue. Our work represents a first step toward evaluating and developing language models that generate new ideas derived from the scientific literature