AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
This work addresses the challenge of manual architecture design in relation classification for NLP researchers, though it is incremental as it builds on existing BERT and NAS methods.
The authors tackled the problem of identifying the optimal architecture for BERT-based relation classification models by using neural architecture search to automate design choices, resulting in improved performance over baseline models across seven benchmark tasks.
Although BERT based relation classification (RC) models have achieved significant improvements over the traditional deep learning models, it seems that no consensus can be reached on what is the optimal architecture. Firstly, there are multiple alternatives for entity span identification. Second, there are a collection of pooling operations to aggregate the representations of entities and contexts into fixed length vectors. Third, it is difficult to manually decide which feature vectors, including their interactions, are beneficial for classifying the relation types. In this work, we design a comprehensive search space for BERT based RC models and employ neural architecture search (NAS) method to automatically discover the design choices mentioned above. Experiments on seven benchmark RC tasks show that our method is efficient and effective in finding better architectures than the baseline BERT based RC model. Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.