HOFAR: High-Order Augmentation of Flow Autoregressive Transformers
This work addresses a specific bottleneck in image generation models, representing an incremental advancement in flow-based autoregressive modeling.
The paper tackled the limitation of first-order trajectory modeling in flow autoregressive transformers for image generation by introducing a high-order supervision framework, resulting in measurable improvements in generation quality compared to baseline models.
Flow Matching and Transformer architectures have demonstrated remarkable performance in image generation tasks, with recent work FlowAR [Ren et al., 2024] synergistically integrating both paradigms to advance synthesis fidelity. However, current FlowAR implementations remain constrained by first-order trajectory modeling during the generation process. This paper introduces a novel framework that systematically enhances flow autoregressive transformers through high-order supervision. We provide theoretical analysis and empirical evaluation showing that our High-Order FlowAR (HOFAR) demonstrates measurable improvements in generation quality compared to baseline models. The proposed approach advances the understanding of flow-based autoregressive modeling by introducing a systematic framework for analyzing trajectory dynamics through high-order expansion.