Pixel-Space Diffusion Transformers
For researchers in generative AI and computer vision, this survey provides a structured overview of pixel-space diffusion transformers, highlighting their potential for unified multimodal systems, but it is an incremental review rather than a novel contribution.
This paper reviews Pixel-Space Diffusion Transformers (pDiTs), which model raw pixels directly to overcome limitations of latent diffusion models such as information loss from VAE compression and mismatch between reconstruction and generation objectives. It summarizes representative methods, identifies key challenges, and discusses future directions toward high-fidelity, end-to-end vision foundation models.
Latent diffusion models (LDMs) enable efficient high-resolution image synthesis by denoising in a VAE-compressed latent space. However, fixed visual tokenizers can discard fine textures and structural details, while separate representation and diffusion training creates a mismatch between reconstruction and generation objectives. These limitations have renewed interest in pixel-space diffusion, which models raw pixels directly, removes the VAE bottleneck, and supports end-to-end optimization. This formulation better matches the demands of high-fidelity generation but introduces challenges in high-dimensional modeling, including noise scheduling, loss weighting, token efficiency, and scalable architecture design. Pixel-space modeling also offers a promising basis for unified multimodal systems: raw pixels, text, and task conditions can be represented in a shared token space and jointly processed by a single Transformer, narrowing the gap between visual understanding and generation. This paper reviews Pixel-Space Diffusion Transformers (pDiTs) from the perspectives of model architecture, continuous generative mechanisms, and unified multimodal modeling. We summarize representative methods, identify key technical challenges, and discuss future directions toward high-fidelity, end-to-end vision foundation models that integrate generation and understanding.