How Much Data is Enough? A Statistical Approach with Case Study on Longitudinal Driving Behavior
It addresses a critical resource allocation issue for costly driving behavior research projects, but the approach is incremental as it applies existing statistical techniques to a specific domain problem.
This paper tackles the problem of determining the optimal amount of naturalistic driving data needed to accurately model driver behaviors, proposing a statistical method that uses Gaussian kernel density estimation and Kullback-Leibler divergence, and demonstrates it on a car-following case study with data from the SPMD program, showing it can estimate an appropriate data amount consistent with literature.
Big data has shown its uniquely powerful ability to reveal, model, and understand driver behaviors. The amount of data affects the experiment cost and conclusions in the analysis. Insufficient data may lead to inaccurate models while excessive data waste resources. For projects that cost millions of dollars, it is critical to determine the right amount of data needed. However, how to decide the appropriate amount has not been fully studied in the realm of driver behaviors. This paper systematically investigates this issue to estimate how much naturalistic driving data (NDD) is needed for understanding driver behaviors from a statistical point of view. A general assessment method is proposed using a Gaussian kernel density estimation to catch the underlying characteristics of driver behaviors. We then apply the Kullback-Liebler divergence method to measure the similarity between density functions with differing amounts of NDD. A max-minimum approach is used to compute the appropriate amount of NDD. To validate our proposed method, we investigated the car-following case using NDD collected from the University of Michigan Safety Pilot Model Deployment (SPMD) program. We demonstrate that from a statistical perspective, the proposed approach can provide an appropriate amount of NDD capable of capturing most features of the normal car-following behavior, which is consistent with the experiment settings in many literatures.