Experimental study on surveillance video-based indoor occupancy measurement with occupant-centric control
This work addresses the need for stable occupancy data to improve HVAC efficiency in smart buildings, representing an incremental advance with domain-specific impact.
This paper tackles the problem of inaccurate vision-based occupancy measurement for smart building HVAC control by comparing detection-only, tracking-based, and LLM-enhanced methods, finding that the YOLOv8+DeepSeek pipeline achieves 0.8824 accuracy and 0.9320 F1-score, and when integrated into HVAC control, yields 17.94% energy savings.
Accurate occupancy information is essential for closed-loop occupant-centric control (OCC) in smart buildings. However, existing vision-based occupancy measurement methods often struggle to provide stable and accurate measurements in real indoor environments, and their implications for downstream HVAC control remain insufficiently studied. To achieve Net Zero emissions by 2050, this paper presents an experimental study of large language models (LLMs)-enhanced vision-based indoor occupancy measurement and its impact on OCC-enabled HVAC operation. Detection-only, tracking-based, and LLM-based refinement pipelines are compared under identical conditions using real surveillance data collected from a research laboratory in China, with frame-level manual ground-truth annotations. Results show that tracking-based methods improve temporal stability over detection-only measurement, while LLM-based refinement further improves occupancy measurement performance and reduces false unoccupied prediction. The best-performing pipeline, YOLOv8+DeepSeek, achieves an accuracy of 0.8824 and an F1-score of 0.9320. This pipeline is then integrated into an HVAC supervisory model predictive control framework in OpenStudio-EnergyPlus. Experimental results demonstrate that the proposed framework can support more efficient OCC operation, achieving a substantial HVAC energy-saving potential of 17.94%. These findings provide an effective methodology and practical foundation for future research in AI-enhanced smart building operations.