Neuro-symbolic Syntactic Parsing: Shaping a Neural Network with the CYK Algorithm
This work offers a new approach to neuro-symbolic methodologies by directly embedding algorithms into neural networks, which could benefit researchers working on interpretable and efficient parsing for specific grammar types.
This paper demonstrates the direct injection of the CYK algorithm into a neural network architecture, named CYKNN, for parsing context-free grammars. The proposed CYKNN architecture outperforms LLMs with over 20B parameters in in-context learning and smaller LoRA-tuned LLMs on a simple grammar with 4 variations.
In this paper, we show the possibility of a direct injection of algorithms into neural network architecture. We focus on a complex algorithm, that is, Cocke-Youger-Kasami (CYK) for parsing context-free grammars in Chomsky Normal Form and we propose CYKNN, a simple recurrent neural network architecture for encoding the CYK algorithm in trainable matrix-vector multiplications.We experimented with a very simple grammar with 4 variations showing that our approach outperforms existing LLMs with more than 20B parameters with an in-context learning setting and smaller LLMs of the Qwen family fine-tuned with LoRA. Our attempt paves the way to a different approach to neuro-symbolic methodologies.