PCA-Based Missing Information Imputation for Real-Time Crash Likelihood Prediction Under Imbalanced Data
This work addresses missing data and imbalance issues in traffic safety prediction, but it is incremental as it applies existing PCA methods and standard imbalance solutions to this domain.
The paper tackles missing data imputation and class imbalance in real-time crash likelihood prediction by employing PCA-based methods and solutions like cost-sensitive learning and SMOTE, resulting in PPCA and VBPCA outperforming other imputation methods in RMSE and improving classifier sensitivity.
The real-time crash likelihood prediction has been an important research topic. Various classifiers, such as support vector machine (SVM) and tree-based boosting algorithms, have been proposed in traffic safety studies. However, few research focuses on the missing data imputation in real-time crash likelihood prediction, although missing values are commonly observed due to breakdown of sensors or external interference. Besides, classifying imbalanced data is also a difficult problem in real-time crash likelihood prediction, since it is hard to distinguish crash-prone cases from non-crash cases which compose the majority of the observed samples. In this paper, principal component analysis (PCA) based approaches, including LS-PCA, PPCA, and VBPCA, are employed for imputing missing values, while two kinds of solutions are developed to solve the problem in imbalanced data. The results show that PPCA and VBPCA not only outperform LS-PCA and other imputation methods (including mean imputation and k-means clustering imputation), in terms of the root mean square error (RMSE), but also help the classifiers achieve better predictive performance. The two solutions, i.e., cost-sensitive learning and synthetic minority oversampling technique (SMOTE), help improve the sensitivity by adjusting the classifiers to pay more attention to the minority class.