Feature Weight Tuning for Recursive Neural Networks
This work addresses a specific bottleneck in recursive neural networks for researchers in natural language processing or compositional AI, though it appears incremental as an enhancement to existing architectures.
The paper tackles the problem of automatically identifying and emphasizing important information in recursive neural networks by proposing two models (WNN and BENN) that control unit contributions to higher-level representations, achieving significant improvements over standard neural models.
This paper addresses how a recursive neural network model can automatically leave out useless information and emphasize important evidence, in other words, to perform "weight tuning" for higher-level representation acquisition. We propose two models, Weighted Neural Network (WNN) and Binary-Expectation Neural Network (BENN), which automatically control how much one specific unit contributes to the higher-level representation. The proposed model can be viewed as incorporating a more powerful compositional function for embedding acquisition in recursive neural networks. Experimental results demonstrate the significant improvement over standard neural models.