A new approach to evaluating legibility: Comparing legibility frameworks using framework-independent robot motion trajectories
This work addresses a methodological bottleneck for researchers in human-robot interaction by providing a more efficient way to compare legibility frameworks, though it is incremental as it focuses on evaluation rather than new legibility generation.
The paper tackles the problem of inefficient data collection in benchmarking legible robot motion trajectories by proposing a novel evaluation method that uses framework-independent trajectories, enabling comparisons of 10 legibility frameworks across 2 scenarios with improved data efficiency.
Robots that share an environment with humans may communicate their intent using a variety of different channels. Movement is one of these channels and, particularly in manipulation tasks, intent communication via movement is called legibility. It alters a robot's trajectory to make it intent expressive. Here we propose a novel evaluation method that improves the data efficiency of collected experimental data when benchmarking approaches generating such legible behavior. The primary novelty of the proposed method is that it uses trajectories that were generated independently of the framework being tested. This makes evaluation easier, enables N-way comparisons between approaches, and allows easier comparison across papers. We demonstrate the efficiency of the new evaluation method by comparing 10 legibility frameworks in 2 scenarios. The paper, thus, provides readers with (1) a novel approach to investigate and/or benchmark legibility, (2) an overview of existing frameworks, (3) an evaluation of 10 legibility frameworks (from 6 papers), and (4) evidence that viewing angle and trajectory progression matter when users evaluate the legibility of a motion.