Stephen D. Boyles

Cesar N. Yahia, Shannon E. Scott, Stephen D. Boyles et al.

Unmanned aerial vehicles (UAVs) provide a novel means of extracting road and traffic information from video data. In particular, by analyzing objects in a video frame, UAVs can detect traffic characteristics and road incidents. Leveraging the mobility and detection capabilities of UAVs, we investigate a navigation algorithm that seeks to maximize information on the road/traffic state under non-recurrent congestion. We propose an active exploration framework that (1) assimilates UAV observations with speed-density sensor data, (2) quantifies uncertainty on the road/traffic state, and (3) adaptively navigates the UAV to minimize this uncertainty. The navigation algorithm uses the A-optimal information measure (mean uncertainty), and it depends on covariance matrices generated by a dual state ensemble Kalman filter (EnKF). In the EnKF procedure, since observations are a nonlinear function of the incident state variables, we use diagnostic variables that represent model predicted measurements. We also present a state update procedure that maintains a monotonic relationship between incident parameters and measurements. We compare the traffic/incident state estimates resulting from the UAV navigation-estimation procedure against corresponding estimates that do not use targeted UAV observations. Our results indicate that UAVs aid in detection of incidents under congested conditions where speed-density data are not informative.

Hillel Bar-Gera, Stephen D. Boyles, Liron Ravner

Vickrey's classic single-bottleneck departure time choice equilibrium model exhibits instability under many plausible day-to-day learning dynamics. Such instability is not observed in reality -- does this difference stem from the day-to-day dynamics or from one of the simplifying assumptions of the basic model? This paper explores a variant of the basic model with a continuous distribution of schedule delay parameters which we intuitively expect to have more favorable stability properties. To attain tractability we assume a monotonic relationship between earliness and lateness parameters. We first verify the existence and uniqueness of the equilibrium solution for this model. We then study a broad class of day-to-day dynamics satisfying local pressure and order preservation conditions. Our main contribution is a formal proof that, surprisingly, all such day-to-day dynamics in this context are unstable.

Venktesh Pandey, Evana Wang, Stephen D. Boyles

This article develops a deep reinforcement learning (Deep-RL) framework for dynamic pricing on managed lanes with multiple access locations and heterogeneity in travelers' value of time, origin, and destination. This framework relaxes assumptions in the literature by considering multiple origins and destinations, multiple access locations to the managed lane, en route diversion of travelers, partial observability of the sensor readings, and stochastic demand and observations. The problem is formulated as a partially observable Markov decision process (POMDP) and policy gradient methods are used to determine tolls as a function of real-time observations. Tolls are modeled as continuous and stochastic variables, and are determined using a feedforward neural network. The method is compared against a feedback control method used for dynamic pricing. We show that Deep-RL is effective in learning toll policies for maximizing revenue, minimizing total system travel time, and other joint weighted objectives, when tested on real-world transportation networks. The Deep-RL toll policies outperform the feedback control heuristic for the revenue maximization objective by generating revenues up to 9.5% higher than the heuristic and for the objective minimizing total system travel time (TSTT) by generating TSTT up to 10.4% lower than the heuristic. We also propose reward shaping methods for the POMDP to overcome the undesired behavior of toll policies, like the jam-and-harvest behavior of revenue-maximizing policies. Additionally, we test transferability of the algorithm trained on one set of inputs for new input distributions and offer recommendations on real-time implementations of Deep-RL algorithms. The source code for our experiments is available online at https://github.com/venktesh22/ExpressLanes_Deep-RL