Discriminative Training: Learning to Describe Video with Sentences, from Video Described with Sentences
This work addresses video description generation for AI and computer vision applications, offering an incremental improvement over existing methods by enhancing performance in data-limited scenarios.
The authors tackled the problem of learning word meanings from video clips to generate sentential descriptions, introducing a discriminative training method that outperforms maximum likelihood approaches, especially with smaller training sets, by leveraging negative labels for better constraints.
We present a method for learning word meanings from complex and realistic video clips by discriminatively training (DT) positive sentential labels against negative ones, and then use the trained word models to generate sentential descriptions for new video. This new work is inspired by recent work which adopts a maximum likelihood (ML) framework to address the same problem using only positive sentential labels. The new method, like the ML-based one, is able to automatically determine which words in the sentence correspond to which concepts in the video (i.e., ground words to meanings) in a weakly supervised fashion. While both DT and ML yield comparable results with sufficient training data, DT outperforms ML significantly with smaller training sets because it can exploit negative training labels to better constrain the learning problem.