Self-supervised learning of video representations from a child's perspective
This work addresses the nature vs. nurture question in AI by showing that generic algorithms can learn temporal aspects from child-like visual data, potentially advancing developmental AI and video understanding, though it is incremental as it builds on existing SSL methods applied to a new dataset.
The researchers tackled whether temporal aspects of a child's internal world model can be learned from egocentric video data using generic self-supervised learning algorithms, and found that video models trained on longitudinal headcam recordings from a child (6-31 months) outperformed image-based models in learning action concepts from few labeled examples, scaling with data size, and achieving emergent video interpolation and more accurate object representations.
Children learn powerful internal models of the world around them from a few years of egocentric visual experience. Can such internal models be learned from a child's visual experience with highly generic learning algorithms or do they require strong inductive biases? Recent advances in collecting large-scale, longitudinal, developmentally realistic video datasets and generic self-supervised learning (SSL) algorithms are allowing us to begin to tackle this nature vs. nurture question. However, existing work typically focuses on image-based SSL algorithms and visual capabilities that can be learned from static images (e.g. object recognition), thus ignoring temporal aspects of the world. To close this gap, here we train self-supervised video models on longitudinal, egocentric headcam recordings collected from a child over a two year period in their early development (6-31 months). The resulting models are highly effective at facilitating the learning of action concepts from a small number of labeled examples; they have favorable data size scaling properties; and they display emergent video interpolation capabilities. Video models also learn more accurate and more robust object representations than image-based models trained with the exact same data. These results suggest that important temporal aspects of a child's internal model of the world may be learnable from their visual experience using highly generic learning algorithms and without strong inductive biases.