From Visual to Acoustic Question Answering
This addresses the problem of acoustic reasoning for researchers in AI and audio processing, but it is incremental as it adapts existing methods from visual to acoustic domains.
The authors introduced Acoustic Question Answering (AQA), a new task for acoustic reasoning that involves analyzing scenes of elementary sounds to answer relational questions, and provided preliminary results using adapted visual reasoning models like FiLM and MAC.
We introduce the new task of Acoustic Question Answering (AQA) to promote research in acoustic reasoning. The AQA task consists of analyzing an acoustic scene composed by a combination of elementary sounds and answering questions that relate the position and properties of these sounds. The kind of relational questions asked, require that the models perform non-trivial reasoning in order to answer correctly. Although similar problems have been extensively studied in the domain of visual reasoning, we are not aware of any previous studies addressing the problem in the acoustic domain. We propose a method for generating the acoustic scenes from elementary sounds and a number of relevant questions for each scene using templates. We also present preliminary results obtained with two models (FiLM and MAC) that have been shown to work for visual reasoning.