ViSTa Dataset: Do vision-language models understand sequential tasks?
This addresses the challenge of using VLMs for reward models in reinforcement learning for sequential tasks, which is incremental as it extends existing VLM applications to more complex scenarios.
The authors tackled the problem of evaluating whether vision-language models (VLMs) can understand sequential tasks beyond goal-oriented ones, introducing the ViSTa dataset with over 4,000 videos and finding that only GPT-4o achieved non-trivial performance while others failed.
Using vision-language models (VLMs) as reward models in reinforcement learning holds promise for reducing costs and improving safety. So far, VLM reward models have only been used for goal-oriented tasks, where the agent must reach a particular final outcome. We explore VLMs' potential to supervise tasks that cannot be scored by the final state alone. To this end, we introduce ViSTa, a dataset for evaluating Vision-based understanding of Sequential Tasks. ViSTa comprises over 4,000 videos with step-by-step descriptions in virtual home, Minecraft, and real-world environments. Its novel hierarchical structure -- basic single-step tasks composed into more and more complex sequential tasks -- allows a fine-grained understanding of how well VLMs can judge tasks with varying complexity. To illustrate this, we use ViSTa to evaluate state-of-the-art VLMs, including CLIP, ViCLIP, and GPT-4o. We find that, while they are all good at object recognition, they fail to understand sequential tasks, with only GPT-4o achieving non-trivial performance.