Empowering Children to Create AI-Enabled Augmented Reality Experiences
This addresses the need for children to actively create with AI and AR, though it is incremental as it builds on existing AR and AI tools for a specific domain.
The paper tackles the problem of children being consumers rather than creators of AI-enabled AR technologies by introducing Capybara, a visual programming environment that allows children to create and program 3D characters using AI models, with user studies involving 20 children showing it empowers them to author personalized AR experiences.
Despite their potential to enhance children's learning experiences, AI-enabled AR technologies are predominantly used in ways that position children as consumers rather than creators. We introduce Capybara, an AR-based and AI-powered visual programming environment that empowers children to create, customize, and program 3D characters overlaid onto the physical world. Capybara enables children to create virtual characters and accessories using text-to-3D generative AI models, and to animate these characters through auto-rigging and body tracking. In addition, our system employs vision-based AI models to recognize physical objects, allowing children to program interactive behaviors between virtual characters and their physical surroundings. We demonstrate the expressiveness of Capybara through a set of novel AR experiences. We conducted user studies with 20 children in the United States and Argentina. Our findings suggest that Capybara can empower children to harness AI in authoring personalized and engaging AR experiences that seamlessly bridge the virtual and physical worlds.