Exploring Mobile Touch Interaction with Large Language Models
This work addresses mobile text editing inefficiencies for users, but it is incremental as it builds on existing LLM and touch interaction concepts.
The paper tackles the problem of cumbersome text editing with LLMs on mobile devices by introducing touch gestures directly on text, demonstrating feasibility and user-friendliness in a user study with 14 participants, where a length + word indicator was most effective.
Interacting with Large Language Models (LLMs) for text editing on mobile devices currently requires users to break out of their writing environment and switch to a conversational AI interface. In this paper, we propose to control the LLM via touch gestures performed directly on the text. We first chart a design space that covers fundamental touch input and text transformations. In this space, we then concretely explore two control mappings: spread-to-generate and pinch-to-shorten, with visual feedback loops. We evaluate this concept in a user study (N=14) that compares three feedback designs: no visualisation, text length indicator, and length + word indicator. The results demonstrate that touch-based control of LLMs is both feasible and user-friendly, with the length + word indicator proving most effective for managing text generation. This work lays the foundation for further research into gesture-based interaction with LLMs on touch devices.