BERT-Based Combination of Convolutional and Recurrent Neural Network for Indonesian Sentiment Analysis
This work addresses sentiment analysis for Indonesian language users, but it is incremental as it applies existing BERT and hybrid methods to a new dataset.
The paper tackled sentiment analysis for Indonesian text by using BERT to generate contextual word representations, which improved the accuracy of hybrid deep learning models like LSTM-CNN, with BERT-based LSTM-CNN achieving slightly better results than other architectures.
Sentiment analysis is the computational study of opinions and emotions ex-pressed in text. Deep learning is a model that is currently producing state-of-the-art in various application domains, including sentiment analysis. Many researchers are using a hybrid approach that combines different deep learning models and has been shown to improve model performance. In sentiment analysis, input in text data is first converted into a numerical representation. The standard method used to obtain a text representation is the fine-tuned embedding method. However, this method does not pay attention to each word's context in the sentence. Therefore, the Bidirectional Encoder Representation from Transformer (BERT) model is used to obtain text representations based on the context and position of words in sentences. This research extends the previous hybrid deep learning using BERT representation for Indonesian sentiment analysis. Our simulation shows that the BERT representation improves the accuracies of all hybrid architectures. The BERT-based LSTM-CNN also reaches slightly better accuracies than other BERT-based hybrid architectures.