Dajiang Suo

Vindula Jayawardana, Catherine Tang, Sirui Li et al.

Evaluations of Deep Reinforcement Learning (DRL) methods are an integral part of scientific progress of the field. Beyond designing DRL methods for general intelligence, designing task-specific methods is becoming increasingly prominent for real-world applications. In these settings, the standard evaluation practice involves using a few instances of Markov Decision Processes (MDPs) to represent the task. However, many tasks induce a large family of MDPs owing to variations in the underlying environment, particularly in real-world contexts. For example, in traffic signal control, variations may stem from intersection geometries and traffic flow levels. The select MDP instances may thus inadvertently cause overfitting, lacking the statistical power to draw conclusions about the method's true performance across the family. In this article, we augment DRL evaluations to consider parameterized families of MDPs. We show that in comparison to evaluating DRL methods on select MDP instances, evaluating the MDP family often yields a substantially different relative ranking of methods, casting doubt on what methods should be considered state-of-the-art. We validate this phenomenon in standard control benchmarks and the real-world application of traffic signal control. At the same time, we show that accurately evaluating on an MDP family is nontrivial. Overall, this work identifies new challenges for empirical rigor in reinforcement learning, especially as the outcomes of DRL trickle into downstream decision-making.

Dajiang Suo, Vindula Jayawardana, Cathy Wu

An emerging public health application of connected and automated vehicle (CAV) technologies is to reduce response times of emergency medical service (EMS) by indirectly coordinating traffic. Therefore, in this work we study the CAV-assisted corridor clearance for EMS vehicles from a short term deployment perspective. Existing research on this topic often overlooks the impact of EMS vehicle disruptions on regular traffic, assumes 100% CAV penetration, relies on real-time traffic signal timing data and queue lengths at intersections, and makes various assumptions about traffic settings when deriving optimal model-based CAV control strategies. However, these assumptions pose significant challenges for near-term deployment and limit the real-world applicability of such methods. To overcome these challenges and enhance real-world applicability in near-term, we propose a model-free approach employing deep reinforcement learning (DRL) for designing CAV control strategies, showing its reduced overhead in designing and greater scalability and performance compared to model-based methods. Our qualitative analysis highlights the complexities of designing scalable EMS corridor clearance controllers for diverse traffic settings in which DRL controller provides ease of design compared to the model-based methods. In numerical evaluations, the model-free DRL controller outperforms the model-based counterpart by improving traffic flow and even improving EMS travel times in scenarios when a single CAV is present. Across 19 considered settings, the learned DRL controller excels by 25% in reducing the travel time in six instances, achieving an average improvement of 9%. These findings underscore the potential and promise of model-free DRL strategies in advancing EMS response and traffic flow coordination, with a focus on practical near-term deployment.

Dajiang Suo, M. Elena Renda, Jinhua Zhao

When it comes to location-based services (LBS), user privacy protection can be in conflict with security of both users and trips. While LBS providers could adopt privacy preservation mechanisms to obfuscate customer data, the accuracy of vehicle location data and trajectories is crucial for detecting anomalies, especially when machine learning methods are adopted by LBS. This paper aims to tackle this dilemma by evaluating the tradeoff between location privacy and security in LBS. In particular, we investigate the impact of applying location data privacy-preservation techniques on the performance of two detectors, namely a Density-based spatial clustering of applications with noise (DBSCAN), and a Recurrent Neural Network (RNN). The experimental results suggest that, by applying privacy on location data, DBSCAN is more sensitive to Laplace noise than RNN, although they achieve similar detection accuracy on the trip data without privacy preservation. Further experiments reveal that DBSCAN is not scalable to large size datasets containing millions of trips, because of the large number of computations needed for clustering trips. On the other hand, DBSCAN only requires less than 10 percent of the data used by RNN to achieve similar performance when applied to vehicle data without obfuscation, demonstrating that clustering-based methods can be easily applied to small datasets. Based on the results, we recommend usage scenarios of the two types of trajectory anomaly detectors when applying privacy preservation, by taking into account customers' need for privacy, the size of the available vehicle trip data, and real-time constraints of the LBS application.

Dajiang Suo, Baichuan Mo, Jinhua Zhao et al.

Previous work on misbehavior detection and trust management for Vehicle-to-Everything (V2X) communication security is effective in identifying falsified and malicious V2X data. Each vehicle in a given region can be a witness to report on the misbehavior of other nearby vehicles, which will then be added to a "blacklist." However, there may not exist enough witness vehicles that are willing to opt-in in the early stage of connected-vehicle deployment. In this paper, we propose a "whitelisting" approach to V2X security, titled Proof-of-Travel (POT), which leverages the support of roadside infrastructure. Our goal is to transform the power of cryptography techniques embedded within Vehicle-to-Infrastructure (V2I) protocols into game-theoretic mechanisms to incentivize connected-vehicle data sharing and validate data trustworthiness simultaneously. The key idea is to determine the reputation of and the contribution made by a vehicle based on its distance traveled and the information it shared through V2I channels. In particular, the total vehicle miles traveled for a vehicle must be testified by digital signatures signed by each infrastructure component along the path of its movement. While building a chain of proofs of spatial movement creates burdens for malicious vehicles, acquiring proofs does not result in extra costs for normal vehicles, which naturally want to move from the origin to the destination. The POT protocol is used to enhance the security of previous voting-based data validation algorithms for V2I crowdsensing applications. For the POT-enhanced voting, we prove that all vehicles choosing to cheat are not a pure Nash equilibrium using game-theoretic analysis. Simulation results suggest that the POT-enhanced voting is more robust to malicious data.