Application of Rényi and Tsallis Entropies to Topic Modeling Optimization
This work addresses a specific challenge in natural language processing for researchers and practitioners using topic modeling, but it appears incremental as it applies existing entropy concepts to a known bottleneck.
The paper tackles the problem of determining the optimal number of topics in topic modeling by proposing the use of Rényi and Tsallis entropies, with numerical experiments showing that semantic stability plays a crucial role in this optimization.
This is full length article (draft version) where problem number of topics in Topic Modeling is discussed. We proposed idea that Renyi and Tsallis entropy can be used for identification of optimal number in large textual collections. We also report results of numerical experiments of Semantic stability for 4 topic models, which shows that semantic stability play very important role in problem topic number. The calculation of Renyi and Tsallis entropy based on thermodynamics approach.