Yifan Chen

M-A-P Team, Xinrun Du, Yifan Yao et al.

Large language models (LLMs) have demonstrated remarkable proficiency in mainstream academic disciplines such as mathematics, physics, and computer science. However, human knowledge encompasses over 200 specialized disciplines, far exceeding the scope of existing benchmarks. The capabilities of LLMs in many of these specialized fields-particularly in light industry, agriculture, and service-oriented disciplines-remain inadequately evaluated. To address this gap, we present SuperGPQA, a comprehensive benchmark that evaluates graduate-level knowledge and reasoning capabilities across 285 disciplines. Our benchmark employs a novel Human-LLM collaborative filtering mechanism to eliminate trivial or ambiguous questions through iterative refinement based on both LLM responses and expert feedback. Our experimental results reveal significant room for improvement in the performance of current state-of-the-art LLMs across diverse knowledge domains (e.g., the reasoning-focused model DeepSeek-R1 achieved the highest accuracy of 61.82% on SuperGPQA), highlighting the considerable gap between current model capabilities and artificial general intelligence. Additionally, we present comprehensive insights from our management of a large-scale annotation process, involving over 80 expert annotators and an interactive Human-LLM collaborative system, offering valuable methodological guidance for future research initiatives of comparable scope.

Yu Chen, S. Yusef Shafi, Yi-fan Chen

Traffic evacuation plays a critical role in saving lives in devastating disasters such as hurricanes, wildfires, floods, earthquakes, etc. An ability to evaluate evacuation plans in advance for these rare events, including identifying traffic flow bottlenecks, improving traffic management policies, and understanding the robustness of the traffic management policy are critical for emergency management. Given the rareness of such events and the corresponding lack of real data, traffic simulation provides a flexible and versatile approach for such scenarios, and furthermore allows dynamic interaction with the simulated evacuation. In this paper, we build a traffic simulation pipeline to explore the above problems, covering many aspects of evacuation, including map creation, demand generation, vehicle behavior, bottleneck identification, traffic management policy improvement, and results analysis. We apply the pipeline to two case studies in California. The first is Paradise, which was destroyed by a large wildfire in 2018 and experienced catastrophic traffic jams during the evacuation. The second is Mill Valley, which has high risk of wildfire and potential traffic issues since the city is situated in a narrow valley.