cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Papers
This work addresses the need for specialized SIMMC methods to support research scientists in interacting with scientific papers, though it is incremental as it builds on existing datasets and LLM techniques.
The authors tackled the problem of enabling situated and multimodal interactive conversations in scientific papers by introducing the cPAPERS dataset, which includes conversational question-answer pairs from paper reviews grounded in text, equations, figures, and tables, and they presented baseline LLM approaches achieving competitive performance on this new benchmark.
An emerging area of research in situated and multimodal interactive conversations (SIMMC) includes interactions in scientific papers. Since scientific papers are primarily composed of text, equations, figures, and tables, SIMMC methods must be developed specifically for each component to support the depth of inquiry and interactions required by research scientists. This work introduces Conversational Papers (cPAPERS), a dataset of conversational question-answer pairs from reviews of academic papers grounded in these paper components and their associated references from scientific documents available on arXiv. We present a data collection strategy to collect these question-answer pairs from OpenReview and associate them with contextual information from LaTeX source files. Additionally, we present a series of baseline approaches utilizing Large Language Models (LLMs) in both zero-shot and fine-tuned configurations to address the cPAPERS dataset.