Upgrading the Newsroom: An Automated Image Selection System for News Articles
This work addresses the challenge of assisting photo editors in newsrooms by automating image selection, though it appears incremental as it builds on existing text-image retrieval methods with specific enhancements.
The researchers tackled the problem of automatically selecting suitable images for news articles by developing a system that fuses multiple textual sources and handles multilingual inputs, achieving superior performance over existing text-image retrieval methods in a weakly-supervised setting.
We propose an automated image selection system to assist photo editors in selecting suitable images for news articles. The system fuses multiple textual sources extracted from news articles and accepts multilingual inputs. It is equipped with char-level word embeddings to help both modeling morphologically rich languages, e.g. German, and transferring knowledge across nearby languages. The text encoder adopts a hierarchical self-attention mechanism to attend more to both keywords within a piece of text and informative components of a news article. We extensively experiment with our system on a large-scale text-image database containing multimodal multilingual news articles collected from Swiss local news media websites. The system is compared with multiple baselines with ablation studies and is shown to beat existing text-image retrieval methods in a weakly-supervised learning setting. Besides, we also offer insights on the advantage of using multiple textual sources and multilingual data.