TaskLens: Generating Task-Conditioned Scaffolded Interfaces for Learning Professional Creative Software
For novices learning complex creative software, TaskLens provides automatically generated scaffolded interfaces that reduce cognitive load and improve learning, addressing a key challenge in HCI and education.
TaskLens generates task-conditioned scaffolded UIs for professional creative software using LLMs. In a user study with 32 beginners, it reduced perceived task load and improved task performance and domain concept learning in Blender.
Professional creative software has steep learning curves for novices due to complex interfaces, limited guidance, and unfamiliar terminology. To support educators and tool creators in addressing learner challenges, we introduce TaskLens, an LLM-based method that automatically generates task-conditioned scaffolded UIs from natural language task descriptions. Our method uses LLMs to identify workflow stages and domain concepts, select task-relevant tools, generate implementation code, and execute the code to produce scaffolded interfaces. The interfaces surface relevant tools, organize them by workflow stage, link them to domain concepts, and progressively disclose advanced features. We evaluate TaskLens by deploying two LLM-generated scaffolded interfaces in Blender, a professional 3D modeling software. A user study with beginners (n=32) showed that our scaffolded interfaces significantly reduced perceived task load, improved task performance through embedded workflow guidance, and increased domain concept learning in Blender during task execution. A second study with experts (n=8) showed improved task efficiency and potential to create personalized UIs for productivity and creativity.