TactileEval: A Step Towards Automated Fine-Grained Evaluation and Editing of Tactile Graphics
This work addresses the need for actionable repair signals in tactile graphics validation, which is crucial for improving accessibility for BVI learners, though it is incremental as it builds on existing datasets and methods.
The paper tackles the problem of automating fine-grained evaluation and editing of tactile graphics for blind and visually impaired learners by introducing TactileEval, a pipeline that achieves 85.70% test accuracy across 30 tasks and enables targeted corrections via automated editing.
Tactile graphics require careful expert validation before reaching blind and visually impaired (BVI) learners, yet existing datasets provide only coarse holistic quality ratings that offer no actionable repair signal. We present TactileEval, a three-stage pipeline that takes a first step toward automating this process. Drawing on expert free-text comments from the TactileNet dataset, we establish a five-category quality taxonomy; encompassing view angle, part completeness, background clutter, texture separation, and line quality aligned with BANA standards. We subsequently gathered 14,095 structured annotations via Amazon Mechanical Turk, spanning 66 object classes organized into six distinct families. A reproducible ViT-L/14 feature probe trained on this data achieves 85.70% overall test accuracy across 30 different tasks, with consistent difficulty ordering suggesting the taxonomy suggesting the taxonomy captures meaningful perceptual structure. Building on these evaluations, we present a ViT-guided automated editing pipeline that routes classifier scores through family-specific prompt templates to produce targeted corrections via gpt-image-1 image editing. Code, data, and models are available at https://TactileEval.github.io/