Yifei Li

Li S. Yifei, Allen Chang, Chaitanya Malaviya et al.

Evaluating long-form responses to research queries heavily relies on expert annotators, restricting attention to areas like AI where researchers can conveniently enlist colleagues. Yet, research expertise is widespread: survey articles synthesize knowledge distributed across the literature. We introduce ResearchQA, a resource for evaluating LLM systems by distilling survey articles from 75 research fields into 21K queries and 160K rubric items. Each rubric, derived jointly with queries from survey sections, lists query-specific answer evaluation criteria, i.e., citing papers, making explanations, and describing limitations. Assessments by 31 Ph.D. annotators in 8 fields indicate 96% of queries support Ph.D. information needs and 87% of rubric items should be addressed in system responses by a sentence or more. Using our rubrics, we are able to construct an automatic pairwise judge obtaining 74% agreement with expert judgments. We leverage ResearchQA to analyze competency gaps in 18 systems in over 7.6K pairwise evaluations. No parametric or retrieval-augmented system we evaluate exceeds 70% on covering rubric items, and the highest-ranking agentic system shows 75% coverage. Error analysis reveals that the highest-ranking system fully addresses less than 11% of citation rubric items, 48% of limitation items, and 49% of comparison items. We release our data to facilitate more comprehensive multi-field evaluations.

Roman V. Grigorii, Yifei Li, Michael A. Peshkin et al.

This study seeks to understand conditions under which virtual gratings produced via vibrotaction and friction modulation are perceived as similar and to find physical origins in the results. To accomplish this, we developed two single-axis devices, one based on electroadhesion and one based on out-of-plane vibration. The two devices had identical touch surfaces, and the vibrotactile device used a novel closed-loop controller to achieve precise control of out-of-plane plate displacement under varying load conditions across a wide range of frequencies. A first study measured the perceptual intensity equivalence curve of gratings generated under electroadhesion and vibrotaction across the 20-400Hz frequency range. A second study assessed the perceptual similarity between two forms of skin excitation given the same driving frequency and same perceived intensity. Our results indicate that it is largely the out-of-plane velocity that predicts vibrotactile intensity relative to shear forces generated by friction modulation. A high degree of perceptual similarity between gratings generated through friction modulation and through vibrotaction is apparent and tends to scale with actuation frequency suggesting perceptual indifference to the manner of fingerpad actuation in the upper frequency range.