LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Tasks
This addresses the need for better benchmarks to assess AI assistance in dynamic, real-world environments for users relying on interactive AI systems, though it is incremental as it builds on existing multimodal evaluation efforts.
The paper tackles the problem of evaluating Multimodal Large Language Models (MLLMs) for real-time assistance in daily life tasks by introducing LifeEval, a benchmark with 4,075 question-answer pairs, and finds that 26 state-of-the-art MLLMs face substantial challenges in achieving timely and adaptive interaction.
The rapid progress of Multimodal Large Language Models (MLLMs) marks a significant step toward artificial general intelligence, offering great potential for augmenting human capabilities. However, their ability to provide effective assistance in dynamic, real-world environments remains largely underexplored. Existing video benchmarks predominantly assess passive understanding through retrospective analysis or isolated perception tasks, failing to capture the interactive and adaptive nature of real-time user assistance. To bridge this gap, we introduce LifeEval, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective. LifeEval emphasizes three key aspects: task-oriented holistic evaluation, egocentric real-time perception from continuous first-person streams, and human-assistant collaborative interaction through natural dialogues. Constructed via a rigorous annotation pipeline, the benchmark comprises 4,075 high-quality question-answer pairs across 6 core capability dimensions. Extensive evaluations of 26 state-of-the-art MLLMs on LifeEval reveal substantial challenges in achieving timely, effective and adaptive interaction, highlighting essential directions for advancing human-centered interactive intelligence.