Beam-Guided Knowledge Replay for Knowledge-Rich Image Captioning using Vision-Language Model
This work addresses the need for more informative and contextually relevant image captions, though it appears incremental with hybrid methods.
The paper tackles the problem of generating generic image captions by proposing KRCapVLM, a knowledge replay-based framework using vision-language models, which improves caption quality and knowledge recognition accuracy with concrete gains in generalization to unseen concepts.
Generating informative and knowledge-rich image captions remains a challenge for many existing captioning models, which often produce generic descriptions that lack specificity and contextual depth. To address this limitation, we propose KRCapVLM, a knowledge replay-based novel image captioning framework using vision-language model. We incorporate beam search decoding to generate more diverse and coherent captions. We also integrate attention-based modules into the image encoder to enhance feature representation. Finally, we employ training schedulers to improve stability and ensure smoother convergence during training. These proposals accelerate substantial gains in both caption quality and knowledge recognition. Our proposed model demonstrates clear improvements in both the accuracy of knowledge recognition and the overall quality of generated captions. It shows a stronger ability to generalize to previously unseen knowledge concepts, producing more informative and contextually relevant descriptions. These results indicate the effectiveness of our approach in enhancing the model's capacity to generate meaningful, knowledge-grounded captions across a range of scenarios.