Benchmark for Anonymous Video Analytics
This provides a standardized evaluation framework for researchers and practitioners in digital advertising and computer vision, though it is incremental as it builds on existing methods without introducing new algorithms.
The authors tackled the lack of a standard benchmark for evaluating computer vision-based audience measurement in out-of-home advertising by proposing the first benchmark for tasks like localization, counting, and demographics, and used it to compare eight open-source algorithms and two commercial solutions across multiple hardware platforms.
Out-of-home audience measurement aims to count and characterize the people exposed to advertising content in the physical world. While audience measurement solutions based on computer vision are of increasing interest, no commonly accepted benchmark exists to evaluate and compare their performance. In this paper, we propose the first benchmark for digital out-of-home audience measurement that evaluates the vision-based tasks of audience localization and counting, and audience demographics. The benchmark is composed of a novel, dataset captured at multiple locations and a set of performance measures. Using the benchmark, we present an in-depth comparison of eight open-source algorithms on four hardware platforms with GPU and CPU-optimized inferences and of two commercial off-the-shelf solutions for localization, count, age, and gender estimation. This benchmark and related open-source codes are available at http://ava.eecs.qmul.ac.uk.