Frozen LVLMs for Micro-Video Recommendation: A Systematic Study of Feature Extraction and Fusion
For practitioners building micro-video recommender systems, this work provides principled guidelines and a lightweight framework for effectively leveraging off-the-shelf LVLMs.
This paper presents the first systematic empirical study on integrating frozen Large Video Language Models (LVLMs) into micro-video recommendation, comparing feature extraction and fusion strategies. It finds that intermediate hidden states outperform captions, ID embeddings are irreplaceable, and proposes the Dual Feature Fusion (DFF) framework, achieving state-of-the-art performance on two benchmarks.
Frozen Large Video Language Models (LVLMs) are increasingly employed in micro-video recommendation due to their strong multimodal understanding. However, their integration lacks systematic empirical evaluation: practitioners typically deploy LVLMs as fixed black-box feature extractors without systematically comparing alternative representation strategies. To address this gap, we present the first systematic empirical study along two key design dimensions: (i) integration strategies with ID embeddings, specifically replacement versus fusion, and (ii) feature extraction paradigms, comparing LVLM-generated captions with intermediate decoder hidden states. Extensive experiments on representative LVLMs reveal three key principles: (1) intermediate hidden states consistently outperform caption-based representations, as natural-language summarization inevitably discards fine-grained visual semantics crucial for recommendation; (2) ID embeddings capture irreplaceable collaborative signals, rendering fusion strictly superior to replacement; and (3) the effectiveness of intermediate decoder features varies significantly across layers. Guided by these insights, we propose the Dual Feature Fusion (DFF) Framework, a lightweight and plug-and-play approach that adaptively fuses multi-layer representations from frozen LVLMs with item ID embeddings. DFF achieves state-of-the-art performance on two real-world micro-video recommendation benchmarks, consistently outperforming strong baselines and providing a principled approach to integrating off-the-shelf large vision-language models into micro-video recommender systems.