Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math Textbooks
This addresses the challenge for teachers and students in making math education more interactive and personalized, though it is incremental as it builds on existing AR and educational technology.
The authors tackled the problem of creating interactive AR-based math explanations from static textbooks without programming, using OCR and computer vision to extract and manipulate formulas and figures, resulting in a system that enables non-technical users to produce engaging learning experiences as confirmed by user studies.
We introduce Augmented Math, a machine learning-based approach to authoring AR explorable explanations by augmenting static math textbooks without programming. To augment a static document, our system first extracts mathematical formulas and figures from a given document using optical character recognition (OCR) and computer vision. By binding and manipulating these extracted contents, the user can see the interactive animation overlaid onto the document through mobile AR interfaces. This empowers non-technical users, such as teachers or students, to transform existing math textbooks and handouts into on-demand and personalized explorable explanations. To design our system, we first analyzed existing explorable math explanations to identify common design strategies. Based on the findings, we developed a set of augmentation techniques that can be automatically generated based on the extracted content, which are 1) dynamic values, 2) interactive figures, 3) relationship highlights, 4) concrete examples, and 5) step-by-step hints. To evaluate our system, we conduct two user studies: preliminary user testing and expert interviews. The study results confirm that our system allows more engaging experiences for learning math concepts.