Arash Tavakoli

Mehdi Fatemi, Arash Tavakoli · microsoft-research

We present a general convergent class of reinforcement learning algorithms that is founded on two distinct principles: (1) mapping value estimates to a different space using arbitrary functions from a broad class, and (2) linearly decomposing the reward signal into multiple channels. The first principle enables incorporating specific properties into the value estimator that can enhance learning. The second principle, on the other hand, allows for the value function to be represented as a composition of multiple utility functions. This can be leveraged for various purposes, e.g. dealing with highly varying reward scales, incorporating a priori knowledge about the sources of reward, and ensemble learning. Combining the two principles yields a general blueprint for instantiating convergent algorithms by orchestrating diverse mapping functions over multiple reward channels. This blueprint generalizes and subsumes algorithms such as Q-Learning, Log Q-Learning, and Q-Decomposition. In addition, our convergence proof for this general class relaxes certain required assumptions in some of these algorithms. Based on our theory, we discuss several interesting configurations as special cases. Finally, to illustrate the potential of the design space that our theory opens up, we instantiate a particular algorithm and evaluate its performance on the Atari suite.

Maximilian Seitzer, Arash Tavakoli, Dimitrije Antic et al.

Capturing aleatoric uncertainty is a critical part of many machine learning systems. In deep learning, a common approach to this end is to train a neural network to estimate the parameters of a heteroscedastic Gaussian distribution by maximizing the logarithm of the likelihood function under the observed data. In this work, we examine this approach and identify potential hazards associated with the use of log-likelihood in conjunction with gradient-based optimizers. First, we present a synthetic example illustrating how this approach can lead to very poor but stable parameter estimates. Second, we identify the culprit to be the log-likelihood loss, along with certain conditions that exacerbate the issue. Third, we present an alternative formulation, termed $β$-NLL, in which each data point's contribution to the loss is weighted by the $β$-exponentiated variance estimate. We show that using an appropriate $β$ largely mitigates the issue in our illustrative example. Fourth, we evaluate this approach on a range of domains and tasks and show that it achieves considerable improvements and performs more robustly concerning hyperparameters, both in predictive RMSE and log-likelihood criteria.

Shiva Azimi, Arash Tavakoli

Cycling plays an important role in sustainable urban mobility, yet how people perceive cycling infrastructure varies widely and remains challenging to assess at large spatial scales. Existing research has mainly relied on surveys or short-form social media data and has often focused on individual cities, leaving limited insight into how cycling discussions unfold across broader geographic contexts. This study proposes a multi-scale framework that examines how cycling infrastructure is discussed and evaluated in online public discourse and explores whether sentiment patterns differ between the United States (U.S.) and selected European countries included in the dataset. The analysis draws on a large collection of discussions on a social media platform, namely Reddit, including more than 30,000 posts and over 500,000 associated comments gathered from cycling-focused and geographically defined communities across multiple U.S. states and selected European countries. Using a combination of sentiment analysis, topic modeling, aspect-based classification, and hierarchical statistical modeling, the study evaluates the emotional tone and thematic structure of these discussions and how they vary spatially. Overall sentiment toward cycling is positive in both regions, with slightly higher values observed in the European sample, although differences remain modest. Sentiment tends to become more critical in comment discussions compared to original posts. Topic and aspect analyses show that sentiment is primarily associated with experience-based themes, with most variation occurring within cities rather than between regions. Together, these findings illustrate how discussion-based online data can complement traditional approaches to understanding public perceptions of cycling infrastructure in sustainable urban contexts.

Xiang Guo, Arash Tavakoli, Erin Robartes et al.

As a healthier and more sustainable way of mobility, cycling has been advocated by literature and policy. However, current trends in bicyclist crash fatalities suggest deficiencies in current roadway design in protecting these vulnerable road users. The lack of cycling data is a common challenge for studying bicyclists' safety, behavior, and comfort levels under different design contexts. To understand bicyclists' behavioral and physiological responses in an efficient and safe way, this study uses a bicycle simulator within an immersive virtual environment (IVE). Off-the-shelf sensors are utilized to evaluate bicyclists' cycling performance (speed and lane position) and physiological responses (eye tracking and heart rate (HR)). Participants bike in a simulated virtual environment modeled to scale from a real-world street with a shared bike lane (sharrow) to evaluate how introduction of a bike lane and a protected bike lane with pylons may impact perceptions of safety, as well as behavioral and psycho-physiological responses. Results from 50 participants show that the protected bike lane design received the highest perceived safety rating and exhibited the lowest average cycling speed. Furthermore, both the bike lane and the protected bike lane scenarios show a less dispersed gaze distribution than the as-built sharrow scenario, reflecting a higher gaze focus among bicyclists on the biking task in the bike lane and protected bike lane scenarios, compared to when bicyclists share right of way with vehicles. Additionally, heart rate change point results from the study suggest that creating dedicated zones for bicyclists (bike lanes or protected bike lanes) has the potential to reduce bicyclists' stress levels.

Arash Tavakoli, Arsalan Heydarian

Naturalistic driving data (NDD) can help understand drivers' reactions to each driving scenario and provide personalized context to driving behavior. However, NDD requires a high amount of manual labor to label certain driver's state and behavioral patterns. Unsupervised analysis of NDD can be used to automatically detect different patterns from the driver and vehicle data. In this paper, we propose a methodology to understand changes in driver's physiological responses within different driving patterns. Our methodology first decomposes a driving scenario by using a Bayesian Change Point detection model. We then apply the Latent Dirichlet Allocation method on both driver state and behavior data to detect patterns. We present two case studies in which vehicles were equipped to collect exterior, interior, and driver behavioral data. Four patterns of driving behaviors (i.e., harsh brake, normal brake, curved driving, and highway driving), as well as two patterns of driver's heart rate (HR) (i.e., normal vs. abnormal high HR), and gaze entropy (i.e., low versus high), were detected in these two case studies. The findings of these case studies indicated that among our participants, the drivers' HR had a higher fraction of abnormal patterns during harsh brakes, accelerating and curved driving. Additionally, free-flow driving with close to zero accelerations on the highway was accompanied by more fraction of normal HR as well as a lower gaze entropy pattern. With the proposed methodology we can better understand variations in driver's psychophysiological states within different driving scenarios. The findings of this work, has the potential to guide future autonomous vehicles to take actions that are fit to each specific driver.

Arash Tavakoli, Shashwat Kumar, Mehdi Boukhechba et al.

Integrating driver, in-cabin, and outside environment's contextual cues into the vehicle's decision making is the centerpiece of semi-automated vehicle safety. Multiple systems have been developed for providing context to the vehicle, which often rely on video streams capturing drivers' physical and environmental states. While video streams are a rich source of information, their ability in providing context can be challenging in certain situations, such as low illuminance environments (e.g., night driving), and they are highly privacy-intrusive. In this study, we leverage passive sensing through smartwatches for classifying elements of driving context. Specifically, through using the data collected from 15 participants in a naturalistic driving study, and by using multiple machine learning algorithms such as random forest, we classify driver's activities (e.g., using phone and eating), outside events (e.g., passing intersection and changing lane), and outside road attributes (e.g., driving in a city versus a highway) with an average F1 score of 94.55, 98.27, and 97.86 % respectively, through 10-fold cross-validation. Our results show the applicability of multimodal data retrieved through smart wearable devices in providing context in real-world driving scenarios and pave the way for a better shared autonomy and privacy-aware driving data-collection, analysis, and feedback for future autonomous vehicles.