HSTU-BLaIR: Lightweight Contrastive Text Embedding for Generative Recommender
This work addresses the need for compute-efficient recommendation systems in e-commerce, though it is incremental as it builds on existing HSTU and contrastive embedding methods.
The paper tackles the problem of improving generative recommender systems by integrating lightweight contrastive text embeddings to enrich item representations with semantic signals from textual metadata, achieving better performance than a state-of-the-art OpenAI embedding-based variant on most metrics across e-commerce datasets.
Recent advances in recommender systems have underscored the complementary strengths of generative modeling and pretrained language models. We propose HSTU-BLaIR, a hybrid framework that augments the Hierarchical Sequential Transduction Unit (HSTU)-based generative recommender with BLaIR, a lightweight contrastive text embedding model. This integration enriches item representations with semantic signals from textual metadata while preserving HSTU's powerful sequence modeling capabilities. We evaluate HSTU-BLaIR on two e-commerce datasets: three subsets from the Amazon Reviews 2023 dataset and the Steam dataset. We compare its performance against both the original HSTU-based recommender and a variant augmented with embeddings from OpenAI's state-of-the-art \texttt{text-embedding-3-large} model. Despite the latter being trained on a substantially larger corpus with significantly more parameters, our lightweight BLaIR-enhanced approach -- pretrained on domain-specific data -- achieves better performance in nearly all cases. Specifically, HSTU-BLaIR outperforms the OpenAI embedding-based variant on all but one metric, where it is marginally lower, and matches it on another. These findings highlight the effectiveness of contrastive text embeddings in compute-efficient recommendation settings.