Balancing Image Compression and Generation with Bootstrapped Tokenization
This work addresses token redundancy in image tokenization for generative models, offering a more efficient and scalable paradigm.
SelfBootTok decomposes image information into global and local token groups via self-bootstrapped learning, enabling generators to use only global tokens and reduce computation by ~40% while achieving a new SOTA gFID of 1.56 with 64 tokens.
Despite progress in image tokenization, standard methods encode redundant information by mixing all granularities within each token, thus redundancy persists between tokens. The mix of information of different granularity also complicates the training of generators. This paper introduces SelfBootTok, a method that resolves this by cleanly decomposing information into global and local token groups. Through self-bootstrapped learning, the model predicts local details exclusively from global tokens, shifting the burden of visual details from the generator to the tokenizer. Consequently, our generator is far more efficient, requiring only global tokens and reducing computation by approximately 40%, while delivering superior reconstruction and generation. Moreover, this paradigm scales elegantly: by leveraging more data or parameters to self-supervise local representation learning, SelfBootTok achieves a new state-of-the-art gFID score of 1.56 using only 64 tokens.