Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style
For users who value personal style in writing, this work reveals that post-editing LLM output only partially achieves stylistic alignment, highlighting limitations of current human-AI collaboration.
In a study with 81 participants, post-editing LLM-generated drafts increased stylistic similarity to participants' unassisted writing but still remained closer to LLM text, with a gap between perceived authenticity and measured similarity.
Despite the growing use of large language models (LLMs) for writing tasks, users may hesitate to rely on LLMs when personal style is important. Post-editing LLM-generated drafts or translations is a common collaborative writing strategy, but it remains unclear whether users can effectively reshape LLM-generated text to reflect their personal style. We conduct a pre-registered online study ($n=81$) in which participants post-edit LLM-generated drafts for writing tasks where personal style matters to them. Using embedding-based style similarity metrics, we find that post-editing increases stylistic similarity to participants' unassisted writing and reduces similarity to fully LLM-generated output. However, post-edited text still remains stylistically closer in style to LLM text than to participants' unassisted control text, and it exhibits reduced stylistic diversity compared to unassisted human text. We find a gap between perceived stylistic authenticity and model-measured stylistic similarity, with post-edited text often perceived as representative of participants' personal style despite remaining detectable LLM stylistic traces.