Detecting Pen-In-Air States from Video: A Proof-of-Concept Toward Complementary Handwriting Analysis
For researchers and clinicians assessing dysgraphia, this work provides a preliminary method to capture Pen-Up movements beyond tablet proximity, though it is an incremental step requiring further validation.
This proof-of-concept study investigates whether top-view video can detect pen-in-air (Pen-Up) states during handwriting, achieving an F2 score up to 0.805, suggesting video as a low-cost complement to digitizing tablets for analyzing handwriting dynamics.
Dynamic aspects of handwriting are critical for assessing developmental disorders such as dysgraphia and are typically captured using digitizing tablets. However, tablet-based sensing restricts analysis of Pen-Up behavior to a short proximity range above the writing surface, potentially missing high-lift in-air movements. As a proof of concept, we investigate whether top-view video can provide a complementary source of information for inferring pen-contact states without relying on tablet proximity sensing. We propose an interpretable hybrid pipeline combining pen-tip tracking using a YOLO-based detector with kinematic feature extraction and machine learning classification. A pilot dataset of diverse handwriting videos was manually annotated at the frame level and evaluation used a Leave-One-Video-Out (LOVO) protocol. The method achieved reliable event-level detection of Pen-Up segments, with an F_2 score up to 0.805, consistent with the emphasis on recall in a screening-oriented setting. These results support the feasibility of video-based Pen-Up detection as a low-cost and non-intrusive complement to digitizing tablets, and provide a foundation for future large-scale studies.