Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation
This work addresses the need for efficient and accurate evaluation in scientific domains like chemistry, though it is incremental in its approach to model enhancement.
The paper tackles the problem of enhancing scientific language models for molecule caption generation with minimal training, achieving results that surpass state-of-the-art models. It also introduces a novel atomic-level evaluation method using NLI models, which effectively handles fine-grained reasoning in the chemical domain.
Scientific language models drive research innovation but require extensive fine-tuning on large datasets. This work enhances such models by improving their inference and evaluation capabilities with minimal or no additional training. Focusing on molecule caption generation, we explore post-training synergies between alignment fine-tuning and model merging in a cross-modal setup. We reveal intriguing insights into the behaviour and suitability of such methods while significantly surpassing state-of-the-art models. Moreover, we propose a novel atomic-level evaluation method leveraging off-the-shelf Natural Language Inference (NLI) models for use in the unseen chemical domain. Our experiments demonstrate that our evaluation operates at the right level of granularity, effectively handling multiple content units and subsentence reasoning, while widely adopted NLI methods consistently misalign with assessment criteria.