Auditory Intelligence: Understanding the World Through Sound
This work addresses the problem of limited contextual and causal understanding in auditory AI for researchers and developers, proposing a new framework to advance the field.
The paper tackles the limitation of current auditory intelligence systems, which focus on surface-level recognition, by proposing a conceptual reframing to include perception, reasoning, and interaction, and introduces four new task paradigms to enable more generalizable and explainable auditory understanding.
Recent progress in auditory intelligence has yielded high-performing systems for sound event detection (SED), acoustic scene classification (ASC), automated audio captioning (AAC), and audio question answering (AQA). Yet these tasks remain largely constrained to surface-level recognition-capturing what happened but not why, what it implies, or how it unfolds in context. I propose a conceptual reframing of auditory intelligence as a layered, situated process that encompasses perception, reasoning, and interaction. To instantiate this view, I introduce four cognitively inspired task paradigms-ASPIRE, SODA, AUX, and AUGMENT-those structure auditory understanding across time-frequency pattern captioning, hierarchical event/scene description, causal explanation, and goal-driven interpretation, respectively. Together, these paradigms provide a roadmap toward more generalizable, explainable, and human-aligned auditory intelligence, and are intended to catalyze a broader discussion of what it means for machines to understand sound.