ChainStream: An LLM-based Framework for Unified Synthetic Sensing
This work addresses privacy and development barriers in context sensing for app developers and end-users, representing an incremental improvement by building on LLMs with specific optimizations.
The paper tackles the challenge of developing context sensing programs by proposing a natural language interface to process personal data, easing app development and increasing transparency. It introduces a unified data processing framework and a feedback-guided query optimizer, achieving efficient and precise performance on a benchmark of 133 context sensing tasks.
Many applications demand context sensing to offer personalized and timely services. Yet, developing sensing programs can be challenging for developers and using them is privacy-concerning for end-users. In this paper, we propose to use natural language as the unified interface to process personal data and sense user context, which can effectively ease app development and make the data pipeline more transparent. Our work is inspired by large language models (LLMs) and other generative models, while directly applying them does not solve the problem - letting the model directly process the data cannot handle complex sensing requests and letting the model write the data processing program suffers error-prone code generation. We address the problem with 1) a unified data processing framework that makes context-sensing programs simpler and 2) a feedback-guided query optimizer that makes data query more informative. To evaluate the performance of natural language-based context sensing, we create a benchmark that contains 133 context sensing tasks. Extensive evaluation has shown that our approach is able to automatically solve the context-sensing tasks efficiently and precisely. The code is opensourced at https://github.com/MobileLLM/ChainStream.