NFLAT: Non-Flat-Lattice Transformer for Chinese Named Entity Recognition
This work addresses efficiency issues in Chinese NER for NLP practitioners, offering an incremental improvement over existing hybrid models.
The paper tackles the high memory and computational costs of the Flat-Lattice Transformer (FLAT) in Chinese Named Entity Recognition by proposing NFLAT, which uses a non-flat lattice method called InterFormer to reduce unnecessary attention calculations, resulting in a 50% reduction in memory usage and enabling more extensive lexicons or higher batch training.
Recently, Flat-LAttice Transformer (FLAT) has achieved great success in Chinese Named Entity Recognition (NER). FLAT performs lexical enhancement by constructing flat lattices, which mitigates the difficulties posed by blurred word boundaries and the lack of word semantics. In FLAT, the positions of starting and ending characters are used to connect a matching word. However, this method is likely to match more words when dealing with long texts, resulting in long input sequences. Therefore, it significantly increases the memory and computational costs of the self-attention module. To deal with this issue, we advocate a novel lexical enhancement method, InterFormer, that effectively reduces the amount of computational and memory costs by constructing non-flat lattices. Furthermore, with InterFormer as the backbone, we implement NFLAT for Chinese NER. NFLAT decouples lexicon fusion and context feature encoding. Compared with FLAT, it reduces unnecessary attention calculations in "word-character" and "word-word". This reduces the memory usage by about 50% and can use more extensive lexicons or higher batches for network training. The experimental results obtained on several well-known benchmarks demonstrate the superiority of the proposed method over the state-of-the-art hybrid (character-word) models.