Revisiting MLLM Token Technology through the Lens of Classical Visual Coding
This work addresses the challenge of computational cost and information fidelity in multimodal AI, offering a structured comparison that could benefit researchers in MLLMs and visual coding, though it appears incremental as it builds on existing principles without introducing a new method.
This paper tackles the problem of improving efficiency and robustness in multimodal large language models (MLLMs) by reexamining MLLM token technology through classical visual coding principles, establishing a unified formulation for comparative analysis and synthesizing bidirectional insights to enhance both fields.
Classical visual coding and Multimodal Large Language Model (MLLM) token technology share the core objective - maximizing information fidelity while minimizing computational cost. Therefore, this paper reexamines MLLM token technology, including tokenization, token compression, and token reasoning, through the established principles of long-developed visual coding area. From this perspective, we (1) establish a unified formulation bridging token technology and visual coding, enabling a systematic, module-by-module comparative analysis; (2) synthesize bidirectional insights, exploring how visual coding principles can enhance MLLM token techniques' efficiency and robustness, and conversely, how token technology paradigms can inform the design of next-generation semantic visual codecs; (3) prospect for promising future research directions and critical unsolved challenges. In summary, this study presents the first comprehensive and structured technology comparison of MLLM token and visual coding, paving the way for more efficient multimodal models and more powerful visual codecs simultaneously.