Recall: Empowering Multimodal Embedding for Edge Devices
This addresses the challenge of efficient information recall for mobile users, representing an incremental improvement in optimizing existing methods for edge deployment.
The paper tackles the problem of high resource demands limiting multimodal embedding models on mobile devices by introducing RECALL, an on-device system that achieves high-throughput and accurate retrieval with minimal memory and energy consumption.
Human memory is inherently prone to forgetting. To address this, multimodal embedding models have been introduced, which transform diverse real-world data into a unified embedding space. These embeddings can be retrieved efficiently, aiding mobile users in recalling past information. However, as model complexity grows, so do its resource demands, leading to reduced throughput and heavy computational requirements that limit mobile device implementation. In this paper, we introduce RECALL, a novel on-device multimodal embedding system optimized for resource-limited mobile environments. RECALL achieves high-throughput, accurate retrieval by generating coarse-grained embeddings and leveraging query-based filtering for refined retrieval. Experimental results demonstrate that RECALL delivers high-quality embeddings with superior throughput, all while operating unobtrusively with minimal memory and energy consumption.