Relational Composition in Neural Networks: A Survey and Call to Action
This work identifies a gap in interpretability for neural networks, potentially impacting researchers and practitioners in AI and machine learning, but it is incremental as it builds on existing survey and analysis without new empirical results.
The paper addresses the incomplete understanding of how neural networks combine feature vectors to represent complex relationships, known as relational composition, by surveying existing mechanisms and proposing future research directions.
Many neural nets appear to represent data as linear combinations of "feature vectors." Algorithms for discovering these vectors have seen impressive recent success. However, we argue that this success is incomplete without an understanding of relational composition: how (or whether) neural nets combine feature vectors to represent more complicated relationships. To facilitate research in this area, this paper offers a guided tour of various relational mechanisms that have been proposed, along with preliminary analysis of how such mechanisms might affect the search for interpretable features. We end with a series of promising areas for empirical research, which may help determine how neural networks represent structured data.