COVCOR20 at WNUT-2020 Task 2: An Attempt to Combine Deep Learning and Expert rules
This is an incremental study for NLP researchers, showing limited progress in integrating machine learning and expert rules.
The paper tackled the problem of text classification in WNUT-2020 Task 2 by combining deep learning models and expert rules, finding that integration improved performance in cross-validation but slightly underperformed the best deep learning model on test data.
In the scope of WNUT-2020 Task 2, we developed various text classification systems, using deep learning models and one using linguistically informed rules. While both of the deep learning systems outperformed the system using the linguistically informed rules, we found that through the integration of (the output of) the three systems a better performance could be achieved than the standalone performance of each approach in a cross-validation setting. However, on the test data the performance of the integration was slightly lower than our best performing deep learning model. These results hardly indicate any progress in line of integrating machine learning and expert rules driven systems. We expect that the release of the annotation manuals and gold labels of the test data after this workshop will shed light on these perplexing results.