Ensemble Mask Networks
This addresses a foundational challenge in neural network interpretability and graph modeling, though it appears incremental as it builds on existing masking and pruning techniques.
The study tackled the problem of whether feedforward networks can learn matrix-vector multiplication by introducing flexible masking and pruning mechanisms, enabling networks to approximate such operations and apply them to test dependencies in graph-based models.
Can an $\mathbb{R}^n\rightarrow \mathbb{R}^n$ feedforward network learn matrix-vector multiplication? This study introduces two mechanisms - flexible masking to take matrix inputs, and a unique network pruning to respect the mask's dependency structure. Networks can approximate fixed operations such as matrix-vector multiplication $φ(A,x) \rightarrow Ax$, motivating the mechanisms introduced with applications towards litmus-testing dependencies or interaction order in graph-based models.