It's Time to Get It Right: Improving Analog Clock Reading and Clock-Hand Spatial Reasoning in Vision-Language Models
This work addresses a critical gap in spatial-temporal reasoning for VLMs, specifically their inability to accurately read analog clocks in diverse real-world settings, impacting applications requiring precise time interpretation from visual data.
This paper investigates the surprising difficulty of state-of-the-art Vision-Language Models (VLMs) in reading analog clocks in real-world environments, attributing it to limitations in existing synthetic datasets. The authors introduce TickTockVQA, a human-annotated dataset of real-world analog clocks, and propose Swap-DPO, a fine-tuning framework, which together substantially improve clock reading accuracy and robustness.
Advances in vision-language models (VLMs) have achieved remarkable success on complex multimodal reasoning tasks, leading to the assumption that they should also excel at reading analog clocks. However, contrary to this expectation, our study reveals that reading analog clocks in real-world environments remains a significant challenge for state-of-the-art VLMs. Existing analog clock datasets are largely synthetic or planar with limited stylistic diversity and minimal background context, failing to capture the visual variability of real-world scenes. As a result, VLMs trained on such data exhibit weak spatial-temporal reasoning, frequently confusing the hour and minute hands and struggling under common visual conditions such as occlusion, lighting variation, and cluttered backgrounds. To address this issue, we introduce TickTockVQA, a human-annotated dataset containing analog clocks in diverse real-world scenarios. TickTockVQA provides explicit hour and minute annotations, and includes an AM/PM tag when it is inferable from the visual context. Furthermore, we propose Swap-DPO, a direct preference optimization based fine-tuning framework to align model reasoning toward accurate time interpretation. Experimental results demonstrate that our approach substantially enhances clock reading accuracy and robustness under real-world conditions, establishing a foundation for future research on spatial-temporal reasoning and visual understanding in VLMs.