Unsupervised Visual and Textual Information Fusion in Multimedia Retrieval - A Graph-based Point of View
This work addresses scalable retrieval from growing multimedia collections for end-users, but it is incremental as it builds on existing graph-based methods.
The authors tackled the problem of multimodal information fusion for content-based multimedia retrieval by proposing a unifying graph-based framework that encompasses cross-media similarities and random walk methods, comparing them on three real-world datasets to provide practical guidelines.
Multimedia collections are more than ever growing in size and diversity. Effective multimedia retrieval systems are thus critical to access these datasets from the end-user perspective and in a scalable way. We are interested in repositories of image/text multimedia objects and we study multimodal information fusion techniques in the context of content based multimedia information retrieval. We focus on graph based methods which have proven to provide state-of-the-art performances. We particularly examine two of such methods : cross-media similarities and random walk based scores. From a theoretical viewpoint, we propose a unifying graph based framework which encompasses the two aforementioned approaches. Our proposal allows us to highlight the core features one should consider when using a graph based technique for the combination of visual and textual information. We compare cross-media and random walk based results using three different real-world datasets. From a practical standpoint, our extended empirical analysis allow us to provide insights and guidelines about the use of graph based methods for multimodal information fusion in content based multimedia information retrieval.