Xinpeng Wang

Qiqi Chen, Xinpeng Wang, Philipp Mondorf et al.

Tree of Thoughts (ToT) is a reasoning strategy for Large Language Models (LLMs) that employs a generator to suggest reasoning steps and a discriminator to decide which steps to implement. ToT demonstrates strong performance on reasoning tasks, often surpassing simple methods such as Input-Output (IO) prompting and Chain-of-Thought (CoT) reasoning. However, ToT does not consistently outperform such simpler methods across all models, leaving large knowledge gaps on the conditions under which ToT is most beneficial. In this paper, we analyze the roles of the generator and discriminator separately to better understand the conditions when ToT is beneficial. We find that the generator plays a more critical role than the discriminator in driving the success of ToT. Scaling the generator leads to notable improvements in ToT performance, even when using a smaller model as the discriminator, whereas scaling the discriminator with a fixed generator yields only marginal gains. Our results show that models across different scales exhibit comparable discrimination capabilities, yet differ significantly in their generative performance for ToT.

Beiduo Chen, Xinpeng Wang, Siyao Peng et al.

Human label variation (HLV) is a valuable source of information that arises when multiple human annotators provide different labels for valid reasons. In Natural Language Inference (NLI) earlier approaches to capturing HLV involve either collecting annotations from many crowd workers to represent human judgment distribution (HJD) or use expert linguists to provide detailed explanations for their chosen labels. While the former method provides denser HJD information, obtaining it is resource-intensive. In contrast, the latter offers richer textual information but it is challenging to scale up to many human judges. Besides, large language models (LLMs) are increasingly used as evaluators ("LLM judges") but with mixed results, and few works aim to study HJDs. This study proposes to exploit LLMs to approximate HJDs using a small number of expert labels and explanations. Our experiments show that a few explanations significantly improve LLMs' ability to approximate HJDs with and without explicit labels, thereby providing a solution to scale up annotations for HJD. However, fine-tuning smaller soft-label aware models with the LLM-generated model judgment distributions (MJDs) presents partially inconsistent results: while similar in distance, their resulting fine-tuned models and visualized distributions differ substantially. We show the importance of complementing instance-level distance measures with a global-level shape metric and visualization to more effectively evaluate MJDs against human judgment distributions.

Xinpeng Wang, Songan Zhang, Kuan-Hui Lee et al.

It is important to build a rigorous verification and validation (V&V) process to evaluate the safety of highly automated vehicles (HAVs) before their wide deployment on public roads. In this paper, we propose an interaction-aware framework for HAV safety evaluation which is suitable for some highly-interactive driving scenarios including highway merging, roundabout entering, etc. Contrary to existing approaches where the primary other vehicle (POV) takes predetermined maneuvers, we model the POV as a game-theoretic agent. To capture a wide variety of interactions between the POV and the vehicle under test (VUT), we characterize the interactive behavior using level-k game theory and social value orientation and train a diverse set of POVs using reinforcement learning. Moreover, we propose an adaptive test case sampling scheme based on the Gaussian process regression technique to generate customized and diverse challenging cases. The highway merging is used as the example scenario. We found the proposed method is able to capture a wide range of POV behaviors and achieve better coverage of the failure modes of the VUT compared with other evaluation approaches.

Nauman Sohani, Geunseob Oh, Xinpeng Wang

We propose a novel framework to differentiate between vehicle trajectories originating from human and non-human drivers by constructing a data-driven boundary using parametric signal temporal logic (STL). Such construction allows us to evaluate the trajectories, detect rare-events, and reduce the uncertainty of driver behaviors when it assumes the form of a disturbance in control synthesis and evaluation problems. We train a classifier that separates admissible (i.e. human) examples - which arise from real-world demonstrations - and inadmissible (i.e. non-human) examples that are generated by falsifying specifications synthesized from the same real-world driving data. Proceeding in this fashion allows for finding a reasonable boundary of human behaviors exhibited in real-world driving records. The framework is demonstrated using a case study involving a human-driven vehicle approaching a signalized intersection.

Zi Yang, Xinpeng Wang, Xin Pei et al.

With the recent advancement in environmental sensing, vehicle control and vehicle-infrastructure cooperation technologies, more and more autonomous driving companies start to put their intelligent cars into road test. But in the near future, we will face a heterogeneous traffic with both intelligent connected vehicles and human vehicles. In this paper, we investigated the impacts of four collision avoidance algorithms under different intelligent connected vehicles market penetration rate. A customized simulation platform is built, in which a platoon can be initiated with many key parameters. For every short time interval, the dynamics of vehicles are updated and input in a kinematics model. If a collision occurs, the energy loss is calculated to represent the crash severity. Four collision avoidance algorithms are chosen and compared in terms of the crash rate and severity at different market penetration rate and different locations of the platoon. The results generate interesting debates on the issues of heterogeneous platoon safety.