Bridge Networks: Relating Inputs through Vector-Symbolic Manipulations
This addresses fundamental limitations in deep learning systems for researchers and practitioners, though it appears incremental as it builds on existing concepts.
The authors tackled challenges in deep learning including catastrophic forgetting and dependence on global losses by proposing a novel 'Bridge network' architecture that combines information bottleneck theory with vector-symbolic architectures, showing it can address these specific problems.
Despite rapid progress, current deep learning methods face a number of critical challenges. These include high energy consumption, catastrophic forgetting, dependance on global losses, and an inability to reason symbolically. By combining concepts from information bottleneck theory and vector-symbolic architectures, we propose and implement a novel information processing architecture, the 'Bridge network.' We show this architecture provides unique advantages which can address the problem of global losses and catastrophic forgetting. Furthermore, we argue that it provides a further basis for increasing energy efficiency of execution and the ability to reason symbolically.