MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning
This work addresses the need for better multilingual code representations in software engineering, offering a novel approach that improves performance on key tasks, though it is incremental in building upon existing multilingual training methods.
The paper tackled the problem of multilingual code representation learning by addressing the oversight of language-specific information in existing methods, proposing MetaTPTrans which uses meta learning to generate dynamic parameters for feature extraction, resulting in significant improvements on state-of-the-art baselines for code summarization and code completion tasks.
Representation learning of source code is essential for applying machine learning to software engineering tasks. Learning code representation from a multilingual source code dataset has been shown to be more effective than learning from single-language datasets separately, since more training data from multilingual dataset improves the model's ability to extract language-agnostic information from source code. However, existing multilingual training overlooks the language-specific information which is crucial for modeling source code across different programming languages, while only focusing on learning a unified model with shared parameters among different languages for language-agnostic information modeling. To address this problem, we propose MetaTPTrans, a meta learning approach for multilingual code representation learning. MetaTPTrans generates different parameters for the feature extractor according to the specific programming language type of the input code snippet, enabling the model to learn both language-agnostic and language-specific information with dynamic parameters in the feature extractor. We conduct experiments on the code summarization and code completion tasks to verify the effectiveness of our approach. The results demonstrate the superiority of our approach with significant improvements on state-of-the-art baselines.