Learning to Select Camera Views: Efficient Multiview Understanding at Few Glances
This addresses the problem of computational efficiency for end devices with limited resources in multiview computer vision applications, representing an incremental improvement by optimizing view selection.
The paper tackles the high computational cost of multiview camera setups by proposing a reinforcement learning-based view selection method that selects the best views for processing, achieving promising performance on classification and detection tasks while using only 2 or 3 out of N views, significantly reducing computational costs.
Multiview camera setups have proven useful in many computer vision applications for reducing ambiguities, mitigating occlusions, and increasing field-of-view coverage. However, the high computational cost associated with multiple views poses a significant challenge for end devices with limited computational resources. To address this issue, we propose a view selection approach that analyzes the target object or scenario from given views and selects the next best view for processing. Our approach features a reinforcement learning based camera selection module, MVSelect, that not only selects views but also facilitates joint training with the task network. Experimental results on multiview classification and detection tasks show that our approach achieves promising performance while using only 2 or 3 out of N available views, significantly reducing computational costs. Furthermore, analysis on the selected views reveals that certain cameras can be shut off with minimal performance impact, shedding light on future camera layout optimization for multiview systems. Code is available at https://github.com/hou-yz/MVSelect.