Modular Mechanistic Networks: On Bridging Mechanistic and Phenomenological Models with Deep Neural Networks in Natural Language Processing
This work addresses the integration of theoretical and empirical approaches in NLP, potentially advancing language technology, but it appears incremental as it builds on existing deep learning concepts without introducing new methods.
The paper tackles the divide between theory-driven mechanistic and data-driven phenomenological models in NLP, arguing that deep neural networks inherently combine both perspectives and can help solve complex problems like spatial cognition modeling.
Natural language processing (NLP) can be done using either top-down (theory driven) and bottom-up (data driven) approaches, which we call mechanistic and phenomenological respectively. The approaches are frequently considered to stand in opposition to each other. Examining some recent approaches in deep learning we argue that deep neural networks incorporate both perspectives and, furthermore, that leveraging this aspect of deep learning may help in solving complex problems within language technology, such as modelling language and perception in the domain of spatial cognition.