BERT-CNN: a Hierarchical Patent Classifier Based on a Pre-Trained Language Model
This work addresses the need for accurate patent classification to aid inventors and examiners in intellectual property protection, though it is incremental as it combines existing techniques.
The authors tackled the problem of automatic patent classification by proposing BERT-CNN, a hierarchical classifier based on a pre-trained language model, which achieved 84.3% accuracy, outperforming baseline methods like CNNs and RNNs.
The automatic classification is a process of automatically assigning text documents to predefined categories. An accurate automatic patent classifier is crucial to patent inventors and patent examiners in terms of intellectual property protection, patent management, and patent information retrieval. We present BERT-CNN, a hierarchical patent classifier based on pre-trained language model by training the national patent application documents collected from the State Information Center, China. The experimental results show that BERT-CNN achieves 84.3% accuracy, which is far better than the two compared baseline methods, Convolutional Neural Networks and Recurrent Neural Networks. We didn't apply our model to the third and fourth hierarchical level of the International Patent Classification - "subclass" and "group".The visualization of the Attention Mechanism shows that BERT-CNN obtains new state-of-the-art results in representing vocabularies and semantics. This article demonstrates the practicality and effectiveness of BERT-CNN in the field of automatic patent classification.