Robot Critics that Sweat the Small Stuff
For roboticists, this method enhances closed-loop manipulation by enabling VLMs to judge small visual differences between success and failure, improving policy success rates.
The paper addresses the challenge of fine-grained progress reasoning and subtle failure detection in robot manipulation using vision-language models. By fine-tuning a critic with pairwise progress supervision from policy rollouts, they improve average policy success rate by 11% in real-world tasks and 5.9% in simulation.
Large vision-language models contain several priors about the world and object interactions, making them useful critics during inference to steer robot policies towards success. However, closed-loop robot manipulation requires judging small visual differences between success and failure, which remains a challenge for current VLMs. We introduce a method to fine-tune critics by constructing pairwise progress supervision using success and failure rollouts obtained from a policy. Our fine-tuned critic excels at fine-grained progress reasoning and subtle failure detection, outperforming prior progress reasoning baselines. Additionally, we use an action-conditioned video model to predict the visual effect of several candidate actions sampled from a policy, and show that our critic can correctly identify successful candidates to execute, improving the average policy success rate by 11% across real-world tasks and 5.9% across simulation tasks.