TaxoAlign: Scholarly Taxonomy Generation Using Language Models
This addresses the need for better automated survey generation tools for researchers, though it appears incremental as it builds on existing approaches with a new benchmark and method.
The authors tackled the problem of automatically generating scholarly taxonomies by proposing TaxoAlign, a method that bridges the gap between human-generated and automatically-created taxonomies, and results show it consistently surpasses baselines on nearly all metrics in evaluations on the CS-TaxoBench benchmark.
Taxonomies play a crucial role in helping researchers structure and navigate knowledge in a hierarchical manner. They also form an important part in the creation of comprehensive literature surveys. The existing approaches to automatic survey generation do not compare the structure of the generated surveys with those written by human experts. To address this gap, we present our own method for automated taxonomy creation that can bridge the gap between human-generated and automatically-created taxonomies. For this purpose, we create the CS-TaxoBench benchmark which consists of 460 taxonomies that have been extracted from human-written survey papers. We also include an additional test set of 80 taxonomies curated from conference survey papers. We propose TaxoAlign, a three-phase topic-based instruction-guided method for scholarly taxonomy generation. Additionally, we propose a stringent automated evaluation framework that measures the structural alignment and semantic coherence of automatically generated taxonomies in comparison to those created by human experts. We evaluate our method and various baselines on CS-TaxoBench, using both automated evaluation metrics and human evaluation studies. The results show that TaxoAlign consistently surpasses the baselines on nearly all metrics. The code and data can be found at https://github.com/AvishekLahiri/TaxoAlign.