Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokens
This work democratizes text-to-image generation by providing open-access models that reduce barriers for researchers and developers, though it is incremental in improving tokenizer efficiency and data accessibility.
The paper tackles the difficulty of training image tokenizers and reliance on private datasets in text-to-image generation by introducing TA-TiTok, an efficient tokenizer that integrates text during decoding, and MaskGen models trained on open data, achieving performance comparable to private-data models.
Image tokenizers form the foundation of modern text-to-image generative models but are notoriously difficult to train. Furthermore, most existing text-to-image models rely on large-scale, high-quality private datasets, making them challenging to replicate. In this work, we introduce Text-Aware Transformer-based 1-Dimensional Tokenizer (TA-TiTok), an efficient and powerful image tokenizer that can utilize either discrete or continuous 1-dimensional tokens. TA-TiTok uniquely integrates textual information during the tokenizer decoding stage (i.e., de-tokenization), accelerating convergence and enhancing performance. TA-TiTok also benefits from a simplified, yet effective, one-stage training process, eliminating the need for the complex two-stage distillation used in previous 1-dimensional tokenizers. This design allows for seamless scalability to large datasets. Building on this, we introduce a family of text-to-image Masked Generative Models (MaskGen), trained exclusively on open data while achieving comparable performance to models trained on private data. We aim to release both the efficient, strong TA-TiTok tokenizers and the open-data, open-weight MaskGen models to promote broader access and democratize the field of text-to-image masked generative models.