Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms
This work addresses a domain-specific problem in natural language processing for researchers and practitioners needing improved topic extraction from document collections, representing an incremental advancement over existing methods.
The paper tackles the problem of traditional topic modeling methods failing to capture contextual semantic information by introducing a semantic-driven approach using transformer-based embeddings and clustering algorithms, resulting in more coherent and meaningful topics compared to ChatGPT and traditional methods.
Topic modeling is a powerful technique to discover hidden topics and patterns within a collection of documents without prior knowledge. Traditional topic modeling and clustering-based techniques encounter challenges in capturing contextual semantic information. This study introduces an innovative end-to-end semantic-driven topic modeling technique for the topic extraction process, utilizing advanced word and document embeddings combined with a powerful clustering algorithm. This semantic-driven approach represents a significant advancement in topic modeling methodologies. It leverages contextual semantic information to extract coherent and meaningful topics. Specifically, our model generates document embeddings using pre-trained transformer-based language models, reduces the dimensions of the embeddings, clusters the embeddings based on semantic similarity, and generates coherent topics for each cluster. Compared to ChatGPT and traditional topic modeling algorithms, our model provides more coherent and meaningful topics.