Pre-gen metrics: Predicting caption quality metrics without generating captions
This provides a faster evaluation method for image captioning researchers, but it is incremental as it builds on existing metrics.
The paper tackled the problem of evaluating image caption generation systems by predicting output quality without generating captions, using the probability assigned by the neural model to reference captions, and found that these pre-gen metrics are strongly correlated to standard evaluation metrics.
Image caption generation systems are typically evaluated against reference outputs. We show that it is possible to predict output quality without generating the captions, based on the probability assigned by the neural model to the reference captions. Such pre-gen metrics are strongly correlated to standard evaluation metrics.