A Perceptually-Validated Metric for Crowd Trajectory Quality Evaluation
This work addresses the challenge of assessing simulation quality for researchers and practitioners in crowd modeling, but it is incremental as it integrates existing strategies into a new metric.
The authors tackled the problem of evaluating crowd trajectory quality in simulations by proposing a metric QF that abstracts from reference data and captures perceptual realism features, validated through an online experiment showing high agreement with non-expert users and applied for parameter tuning in crowd motion models.
Simulating crowds requires controlling a very large number of trajectories and is usually performed using crowd motion algorithms for which appropriate parameter values need to be found. The study of the relation between parametric values for simulation techniques and the quality of the resulting trajectories has been studied either through perceptual experiments or by comparison with real crowd trajectories. In this paper, we integrate both strategies. A quality metric, QF, is proposed to abstract from reference data while capturing the most salient features that affect the perception of trajectory realism. QF weights and combines cost functions that are based on several individual, local and global properties of trajectories. These trajectory features are selected from the literature and from interviews with experts. To validate the capacity of QF to capture perceived trajectory quality, we conduct an online experiment that demonstrates the high agreement between the automatic quality score and non-expert users. To further demonstrate the usefulness of QF, we use it in a data-free parameter tuning application able to tune any parametric microscopic crowd simulation model that outputs independent trajectories for characters. The learnt parameters for the tuned crowd motion model maintain the influence of the reference data which was used to weight the terms of QF.