A Domain-Specific Language for LLM-Driven Trigger Generation in Multimodal Data Collection
This addresses the issue of high storage costs and irrelevant data capture in real-time systems like autonomous vehicles and robots, though it is an incremental improvement by combining existing methods (DSL and LLMs) for a known bottleneck.
The paper tackles the problem of passive and costly multimodal data collection by proposing a declarative framework that uses a domain-specific language (DSL) and large language models to generate conditional triggers for selective data capture. The result shows higher generation consistency and lower execution latency compared to unconstrained code generation, while maintaining comparable detection performance in vehicular and robotic tasks.
Data-driven systems depend on task-relevant data, yet data collection pipelines remain passive and indiscriminate. Continuous logging of multimodal sensor streams incurs high storage costs and captures irrelevant data. This paper proposes a declarative framework for intent-driven, on-device data collection that enables selective collection of multimodal sensor data based on high-level user requests. The framework combines natural language interaction with a formally specified domain-specific language (DSL). Large language models translate user-defined requirements into verifiable and composable DSL programs that define conditional triggers across heterogeneous sensors, including cameras, LiDAR, and system telemetry. Empirical evaluation on vehicular and robotic perception tasks shows that the DSL-based approach achieves higher generation consistency and lower execution latency than unconstrained code generation while maintaining comparable detection performance. The structured abstraction supports modular trigger composition and concurrent deployment on resource-constrained edge platforms. This approach replaces passive logging with a verifiable, intent-driven mechanism for multimodal data collection in real-time systems.