Specializing Joint Representations for the task of Product Recommendation
This work addresses retrieval-based product recommendation for e-commerce, but it is incremental as it builds on existing multimodal fusion methods with a modular approach.
The paper tackles product recommendation by proposing a unified product embedding that fuses modality-specific embeddings (text, images, collaborative filtering) at the end of the architecture, achieving good performance on a large dataset in scenarios like cold-start and cross-category recommendations.
We propose a unified product embedded representation that is optimized for the task of retrieval-based product recommendation. To this end, we introduce a new way to fuse modality-specific product embeddings into a joint product embedding, in order to leverage both product content information, such as textual descriptions and images, and product collaborative filtering signal. By introducing the fusion step at the very end of our architecture, we are able to train each modality separately, allowing us to keep a modular architecture that is preferable in real-world recommendation deployments. We analyze our performance on normal and hard recommendation setups such as cold-start and cross-category recommendations and achieve good performance on a large product shopping dataset.