Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings
It addresses the problem of retrieving cooking-related images and recipes for users, but it is incremental as it builds on existing cross-modal retrieval methods.
The paper tackles cross-modal retrieval between images and text in cooking by learning semantic embeddings, achieving improved performance over previous state-of-the-art models on the Recipe1M dataset with nearly 1 million pairs.
Designing powerful tools that support cooking activities has rapidly gained popularity due to the massive amounts of available data, as well as recent advances in machine learning that are capable of analyzing them. In this paper, we propose a cross-modal retrieval model aligning visual and textual data (like pictures of dishes and their recipes) in a shared representation space. We describe an effective learning scheme, capable of tackling large-scale problems, and validate it on the Recipe1M dataset containing nearly 1 million picture-recipe pairs. We show the effectiveness of our approach regarding previous state-of-the-art models and present qualitative results over computational cooking use cases.