Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?
This work addresses the coherence of multimodal AI models for researchers and practitioners, but it is incremental as it provides a dataset and evaluation rather than a new method.
The paper tackled the problem of whether any-to-any generative models achieve greater cross-modal consistency than specialized models, finding that they do not show consistent improvements in pointwise evaluations but exhibit weak consistency in structured analyses.
Any-to-any generative models aim to enable seamless interpretation and generation across multiple modalities within a unified framework, yet their ability to preserve relationships across modalities remains uncertain. Do unified models truly achieve cross-modal coherence, or is this coherence merely perceived? To explore this, we introduce ACON, a dataset of 1,000 images (500 newly contributed) paired with captions, editing instructions, and Q&A pairs to evaluate cross-modal transfers rigorously. Using three consistency criteria-cyclic consistency, forward equivariance, and conjugated equivariance-our experiments reveal that any-to-any models do not consistently demonstrate greater cross-modal consistency than specialized models in pointwise evaluations such as cyclic consistency. However, equivariance evaluations uncover weak but observable consistency through structured analyses of the intermediate latent space enabled by multiple editing operations. We release our code and data at https://github.com/JiwanChung/ACON.