Making Sense of Touch from the Child's View for Contrastive Learning
For developmental psychologists and AI researchers, it provides a dataset and framework to study the role of touch in visual concept learning, though the results are preliminary.
The paper investigates whether touch helps babies learn visual concepts, creating a dataset of 264k touch clips and using it to pretrain models that show touch aids visual learning.
Is the sense of touch a mechanism for human babies' learning of visual concepts? If so, can we quantify its importance, and to what extent do babies rely on their sense of touch for visual learning? To approach these questions in a principled way, we propose a structured coding system for baby-centric touch events, yielding a dataset of 264k two-second clips of touch events coded according to this system. Using this dataset, we pretrain developmentally grounded models that reveal promising insights into the nature of baby learning from touch.