Towards Practicality of Sketch-Based Visual Understanding
It aims to improve sketch-based tools for users and developers, but appears incremental as it builds on existing sketch research to address known bottlenecks.
This thesis addresses the challenge of making sketch-based visual understanding more practical by tackling the issues of sketch abstraction, subjectivity, and the difficulty of collecting fine-grained sketch-photo associations, with the goal of advancing applications like image retrieval and editing.
Sketches have been used to conceptualise and depict visual objects from pre-historic times. Sketch research has flourished in the past decade, particularly with the proliferation of touchscreen devices. Much of the utilisation of sketch has been anchored around the fact that it can be used to delineate visual concepts universally irrespective of age, race, language, or demography. The fine-grained interactive nature of sketches facilitates the application of sketches to various visual understanding tasks, like image retrieval, image-generation or editing, segmentation, 3D-shape modelling etc. However, sketches are highly abstract and subjective based on the perception of individuals. Although most agree that sketches provide fine-grained control to the user to depict a visual object, many consider sketching a tedious process due to their limited sketching skills compared to other query/support modalities like text/tags. Furthermore, collecting fine-grained sketch-photo association is a significant bottleneck to commercialising sketch applications. Therefore, this thesis aims to progress sketch-based visual understanding towards more practicality.