CoSE: Compositional Stroke Embeddings
This work addresses the challenge of generating and auto-completing diagrams for interactive use cases, representing an incremental advance over previous sequence-based models for basic drawings.
The paper tackles the problem of modeling complex free-form structures like diagrams by proposing a generative model that treats drawings as collections of strokes, using a novel autoencoder to embed strokes into a fixed-dimensional latent space and a relational model to capture stroke relationships, resulting in improved modeling of stroke appearance and compositional structure as demonstrated qualitatively and quantitatively.
We present a generative model for complex free-form structures such as stroke-based drawing tasks. While previous approaches rely on sequence-based models for drawings of basic objects or handwritten text, we propose a model that treats drawings as a collection of strokes that can be composed into complex structures such as diagrams (e.g., flow-charts). At the core of the approach lies a novel autoencoder that projects variable-length strokes into a latent space of fixed dimension. This representation space allows a relational model, operating in latent space, to better capture the relationship between strokes and to predict subsequent strokes. We demonstrate qualitatively and quantitatively that our proposed approach is able to model the appearance of individual strokes, as well as the compositional structure of larger diagram drawings. Our approach is suitable for interactive use cases such as auto-completing diagrams. We make code and models publicly available at https://eth-ait.github.io/cose.