The Effect of Multiple Replies for Natural Language Generation Chatbots
This addresses user experience issues in NLG chatbots, but it is incremental as it builds on existing methods by modifying reply presentation.
The study tackled the problem of inappropriate content in Natural Language Generation chatbots by providing multiple replies to improve user experience, finding that users had a better chatting experience with five replies compared to a single reply, with an effect size showing greater benefits from multiple replies than from using an anime avatar.
In this research, by responding to users' utterances with multiple replies to create a group chat atmosphere, we alleviate the problem that Natural Language Generation chatbots might reply with inappropriate content, thus causing a bad user experience. Because according to our findings, users tend to pay attention to appropriate replies and ignore inappropriate replies. We conducted a 2 (single reply vs. five replies) x 2 (anonymous avatar vs. anime avatar) repeated measures experiment to compare the chatting experience in different conditions. The result shows that users will have a better chatting experience when receiving multiple replies at once from the NLG model compared to the single reply. Furthermore, according to the effect size of our result, to improve the chatting experience for NLG chatbots which is single reply and anonymous avatar, providing five replies will have more benefits than setting an anime avatar.