A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code
This addresses a bottleneck for researchers and practitioners applying NLP to software engineering, though it is an incremental improvement.
The paper tackles the problem of rare identifiers causing large vocabulararies in deep learning for source code by proposing an identifier anonymization method, which improves Transformer performance by up to 15% in code completion and bug fixing tasks.
There is an emerging interest in the application of natural language processing models to source code processing tasks. One of the major problems in applying deep learning to software engineering is that source code often contains a lot of rare identifiers, resulting in huge vocabularies. We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and, therefore, allows for easy implementation. We show that the proposed OOV anonymization method significantly improves the performance of the Transformer in two code processing tasks: code completion and bug fixing.