FONTNET: On-Device Font Understanding and Prediction Pipeline
This work addresses font understanding for applications like text customization and image processing, offering privacy and low latency for real-time use, but it is incremental as it builds on existing text recognition and CNN methods.
The paper tackles the problem of understanding and predicting fonts in images by proposing a pipeline with two engines: one for detecting font attributes like style, color, and size, and another for predicting similar fonts, achieving on-device deployment with a worst-case inference time of 30ms and a model size of 4.5MB.
Fonts are one of the most basic and core design concepts. Numerous use cases can benefit from an in depth understanding of Fonts such as Text Customization which can change text in an image while maintaining the Font attributes like style, color, size. Currently, Text recognition solutions can group recognized text based on line breaks or paragraph breaks, if the Font attributes are known multiple text blocks can be combined based on context in a meaningful manner. In this paper, we propose two engines: Font Detection Engine, which identifies the font style, color and size attributes of text in an image and a Font Prediction Engine, which predicts similar fonts for a query font. Major contributions of this paper are three-fold: First, we developed a novel CNN architecture for identifying font style of text in images. Second, we designed a novel algorithm for predicting similar fonts for a given query font. Third, we have optimized and deployed the entire engine On-Device which ensures privacy and improves latency in real time applications such as instant messaging. We achieve a worst case On-Device inference time of 30ms and a model size of 4.5MB for both the engines.