Robust Bidirectional Associative Memory via Regularization Inspired by the Subspace Rotation Algorithm
This work addresses robustness issues in associative memories for applications requiring resilience to noise and attacks, though it is incremental as it builds on existing BAM methods.
The paper tackled the poor robustness and sensitivity to noise and adversarial attacks in Bidirectional Associative Memory (BAM) trained with Bidirectional Backpropagation (B-BP), proposing a novel gradient-free training algorithm (B-SRA) and regularization strategies that significantly improved robustness, with the SAME configuration achieving the strongest resilience in experiments across memory capacities like 50, 100, and 200 associative pairs.
Bidirectional Associative Memory (BAM) trained with Bidirectional Backpropagation (B-BP) often suffers from poor robustness and high sensitivity to noise and adversarial attacks. To address these issues, we propose a novel gradient-free training algorithm, the Bidirectional Subspace Rotation Algorithm (B-SRA), which significantly improves the robustness and convergence behavior of BAM. Through comprehensive experiments, we identify two key principles -- orthogonal weight matrices (OWM) and gradient-pattern alignment (GPA) -- as central to enhancing the robustness of BAM. Motivated by these findings, we introduce new regularization strategies into B-BP, resulting in models with greatly improved resistance to corruption and adversarial perturbations. We further conduct an ablation study across different training strategies to determine the most robust configuration and evaluate BAM's performance under a variety of attack scenarios and memory capacities, including 50, 100, and 200 associative pairs. Among all methods, the SAME configuration, which integrates both OWM and GPA, achieves the strongest resilience. Overall, our results demonstrate that B-SRA and the proposed regularization strategies lead to substantially more robust associative memories and open new directions for building resilient neural architectures.