Performance Comparison of Simple Transformer and Res-CNN-BiLSTM for Cyberbullying Classification
This is an incremental comparison of existing methods for cyberbullying detection, addressing computational efficiency in text classification.
The paper tackled the problem of cyberbullying text classification by comparing a simple transformer network with a Res-CNN-BiLSTM network, finding that the transformer with 0.65 million parameters outperformed the Res-CNN-BiLSTM with 48.82 million parameters in faster training and more generalized metrics.
The task of text classification using Bidirectional based LSTM architectures is computationally expensive and time consuming to train. For this, transformers were discovered which effectively give good performance as compared to the traditional deep learning architectures. In this paper we present a performance based comparison between simple transformer based network and Res-CNN-BiLSTM based network for cyberbullying text classification problem. The results obtained show that transformer we trained with 0.65 million parameters has significantly being able to beat the performance of Res-CNN-BiLSTM with 48.82 million parameters for faster training speeds and more generalized metrics. The paper also compares the 1-dimensional character level embedding network and 100-dimensional glove embedding network with transformer.