Self-Supervised Learning for Visual Summary Identification in Scientific Publications
This work addresses the problem of limited information access for readers of scientific publications by providing an automated method for visual summary identification, which is particularly useful given the scarcity of annotated datasets.
The paper tackles the problem of automatically identifying visual summaries in scientific publications, crucial for managing the exponential growth of scientific literature. They developed a self-supervised learning approach that outperforms state-of-the-art methods in both biomedical and computer science domains without relying on annotated training data.
Providing visual summaries of scientific publications can increase information access for readers and thereby help deal with the exponential growth in the number of scientific publications. Nonetheless, efforts in providing visual publication summaries have been few and far apart, primarily focusing on the biomedical domain. This is primarily because of the limited availability of annotated gold standards, which hampers the application of robust and high-performing supervised learning techniques. To address these problems we create a new benchmark dataset for selecting figures to serve as visual summaries of publications based on their abstracts, covering several domains in computer science. Moreover, we develop a self-supervised learning approach, based on heuristic matching of inline references to figures with figure captions. Experiments in both biomedical and computer science domains show that our model is able to outperform the state of the art despite being self-supervised and therefore not relying on any annotated training data.