NELS -- Never-Ending Learner of Sounds
This addresses the need for scalable sound indexing and retrieval from vast online video archives, though it appears incremental as it builds on existing sound recognition methods.
The authors tackled the problem of automatic sound analysis from unlabeled web videos by proposing NELS, a system that continuously learns sound-language relations and improves recognition models, achieving large-scale evaluation without references.
Sounds are essential to how humans perceive and interact with the world and are captured in recordings and shared on the Internet on a minute-by-minute basis. These recordings, which are predominantly videos, constitute the largest archive of sounds we know. However, most of these recordings have undescribed content making necessary methods for automatic sound analysis, indexing and retrieval. These methods have to address multiple challenges, such as the relation between sounds and language, numerous and diverse sound classes, and large-scale evaluation. We propose a system that continuously learns from the web relations between sounds and language, improves sound recognition models over time and evaluates its learning competency in the large-scale without references. We introduce the Never-Ending Learner of Sounds (NELS), a project for continuously learning of sounds and their associated knowledge, available on line in nels.cs.cmu.edu