Color When It Counts: Grayscale-Guided Online Triggering for Always-On Streaming Video Sensing
This enables practical always-on video sensing on resource-constrained mobile and edge devices by reducing sensing and inference costs.
The paper tackles the problem of high energy costs for continuous RGB video capture in always-on edge/wearable AI systems by proposing a grayscale-always, color-on-demand paradigm, achieving 91.6% of full-color baseline performance with only 8.1% RGB frames.
Always-on sensing is essential for next-generation edge/wearable AI systems, yet continuous high-fidelity RGB video capture remains prohibitively expensive for resource-constrained mobile and edge platforms. We present a new paradigm for efficient streaming video understanding: grayscale-always, color-on-demand. Through preliminary studies, we discover that color is not always necessary. Sparse RGB frames suffice for comparable performance when temporal structure is preserved via continuous grayscale streams. Building on this insight, we propose ColorTrigger, an online training-free trigger that selectively activates color capture based on windowed grayscale affinity analysis. Designed for real-time edge deployment, ColorTrigger uses lightweight quadratic programming to detect chromatic redundancy causally, coupled with credit-budgeted control and dynamic token routing to jointly reduce sensing and inference costs. On streaming video understanding benchmarks, ColorTrigger achieves 91.6% of full-color baseline performance while using only 8.1% RGB frames, demonstrating substantial color redundancy in natural videos and enabling practical always-on video sensing on resource-constrained devices.