Tuning computer vision models with task rewards
This addresses a critical deployment issue for computer vision practitioners, offering a novel method to improve model alignment across diverse tasks, though it builds on existing NLP approaches.
The paper tackles misalignment between model predictions and intended usage in computer vision by adopting reinforcement learning techniques from NLP to align models with task rewards, showing surprising effectiveness across tasks like object detection, panoptic segmentation, colorization, and image captioning.
Misalignment between model predictions and intended usage can be detrimental for the deployment of computer vision models. The issue is exacerbated when the task involves complex structured outputs, as it becomes harder to design procedures which address this misalignment. In natural language processing, this is often addressed using reinforcement learning techniques that align models with a task reward. We adopt this approach and show its surprising effectiveness across multiple computer vision tasks, such as object detection, panoptic segmentation, colorization and image captioning. We believe this approach has the potential to be widely useful for better aligning models with a diverse range of computer vision tasks.