NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classification
This work addresses data generation challenges for code comment classification in a specific programming language, representing an incremental improvement.
The authors tackled the problem of generating synthetic data for code comment classification in C by combining a neuro-symbolic workflow with an LLM agent, resulting in a neural network model achieving a Macro-F1 score of 91.412% with a 1.033% improvement after data augmentation.
We present a neuro-symbolic (NeSy) workflow combining a symbolic-based learning technique with a large language model (LLM) agent to generate synthetic data for code comment classification in the C programming language. We also show how generating controlled synthetic data using this workflow fixes some of the notable weaknesses of LLM-based generation and increases the performance of classical machine learning models on the code comment classification task. Our best model, a Neural Network, achieves a Macro-F1 score of 91.412% with an increase of 1.033% after data augmentation.