Lucy: edgerunning agentic web search on mobile with machine generated task vectors
This work addresses the problem of constrained capacity in small language models for knowledge-intensive tasks, offering a novel approach that could enhance efficiency in mobile or edge computing applications.
The paper tackles the limitation of small language models in knowledge-intensive tasks by proposing a new paradigm where the model's internal reasoning is viewed as a dynamic task vector machine, enabling a 1.7B-parameter model to achieve 78.3% accuracy on the SimpleQA benchmark, rivaling much larger models.
Small language models (SLMs) are inherently limited in knowledge-intensive tasks due to their constrained capacity. While test-time computation offers a path to enhanced performance, most approaches treat reasoning as a fixed or heuristic process. In this work, we propose a new paradigm: viewing the model's internal reasoning, delimited by <think> and </think> tags, as a dynamic task vector machine. Rather than treating the content inside these tags as a mere trace of thought, we interpret the generation process itself as a mechanism through which the model \textbf{constructs and refines its own task vectors} on the fly. We developed a method to optimize this dynamic task vector machine through RLVR and successfully trained an agentic web-search model. We present Lucy, a 1.7B-parameter SLM that leverages this dynamic reasoning mechanism with MCP integration to achieve 78.3% accuracy on the SimpleQA benchmark, performing on par with much larger models such as DeepSeek-V3. This demonstrates that small models can rival large ones when equipped with structured, self-constructed task reasoning.