Human-AI Collaborative Taxonomy Construction: A Case Study in Profession-Specific Writing Assistants
This work addresses the need for more effective LLM-powered writing assistance tailored to domain-specific requirements, such as for business professionals, but it appears incremental as it builds on existing human-AI collaboration methods without claiming major breakthroughs.
The paper tackled the problem of LLMs' limited understanding of domain-specific writing nuances in business contexts by proposing a human-AI collaborative taxonomy development approach, which integrates iterative expert feedback and interactions to refine guidelines for writing assistants, with larger-scale experiments planned for validation.
Large Language Models (LLMs) have assisted humans in several writing tasks, including text revision and story generation. However, their effectiveness in supporting domain-specific writing, particularly in business contexts, is relatively less explored. Our formative study with industry professionals revealed the limitations in current LLMs' understanding of the nuances in such domain-specific writing. To address this gap, we propose an approach of human-AI collaborative taxonomy development to perform as a guideline for domain-specific writing assistants. This method integrates iterative feedback from domain experts and multiple interactions between these experts and LLMs to refine the taxonomy. Through larger-scale experiments, we aim to validate this methodology and thus improve LLM-powered writing assistance, tailoring it to meet the unique requirements of different stakeholder needs.