ProtoSnap: Prototype Alignment for Cuneiform Signs
This work addresses a domain-specific problem for researchers in archaeology and paleography by providing a novel method to analyze cuneiform sign variations, though it is incremental in applying existing generative models to this niche area.
The paper tackles the problem of modeling the fine-grained internal structure of cuneiform signs, which prior automated techniques treated as categorical, by presenting an unsupervised approach that aligns prototype skeletons to photographed signs and generates synthetic data, resulting in significantly boosted performance for cuneiform sign recognition, especially for rare signs.
The cuneiform writing system served as the medium for transmitting knowledge in the ancient Near East for a period of over three thousand years. Cuneiform signs have a complex internal structure which is the subject of expert paleographic analysis, as variations in sign shapes bear witness to historical developments and transmission of writing and culture over time. However, prior automated techniques mostly treat sign types as categorical and do not explicitly model their highly varied internal configurations. In this work, we present an unsupervised approach for recovering the fine-grained internal configuration of cuneiform signs by leveraging powerful generative models and the appearance and structure of prototype font images as priors. Our approach, ProtoSnap, enforces structural consistency on matches found with deep image features to estimate the diverse configurations of cuneiform characters, snapping a skeleton-based template to photographed cuneiform signs. We provide a new benchmark of expert annotations and evaluate our method on this task. Our evaluation shows that our approach succeeds in aligning prototype skeletons to a wide variety of cuneiform signs. Moreover, we show that conditioning on structures produced by our method allows for generating synthetic data with correct structural configurations, significantly boosting the performance of cuneiform sign recognition beyond existing techniques, in particular over rare signs. Our code, data, and trained models are available at the project page: https://tau-vailab.github.io/ProtoSnap/