Aspect and Opinion Term Extraction for Hotel Reviews using Transfer Learning and Auxiliary Labels
This work addresses a domain-specific problem for hotel review analysis in a low-resource language, with incremental improvements over existing methods.
The study tackled aspect and opinion term extraction from informal hotel reviews in bahasa Indonesia using transfer learning with BERT, achieving up to 2% higher token-level F1-score than a Bi-LSTM model with far fewer training epochs (3 vs. 200) and proposing a CRF layer with auxiliary labels that further improved performance.
Aspect and opinion term extraction is a critical step in Aspect-Based Sentiment Analysis (ABSA). Our study focuses on evaluating transfer learning using pre-trained BERT (Devlin et al., 2018) to classify tokens from hotel reviews in bahasa Indonesia. The primary challenge is the language informality of the review texts. By utilizing transfer learning from a multilingual model, we achieved up to 2% difference on token level F1-score compared to the state-of-the-art Bi-LSTM model with fewer training epochs (3 vs. 200 epochs). The fine-tuned model clearly outperforms the Bi-LSTM model on the entity level. Furthermore, we propose a method to include CRF with auxiliary labels as an output layer for the BERT-based models. The CRF addition further improves the F1-score for both token and entity level.