Data Processing Techniques for Modern Multimodal Models
This is an incremental review aimed at multimodal model developers, offering guidance on effective data processing techniques.
The paper provides a comprehensive review of common data processing techniques for modern multimodal models, focusing on diffusion models and MLLMs, summarizing them into four categories to guide developers.
Data processing plays an significant role in current multimodal model training. In this paper. we provide an comprehensive review of common data processing techniques used in modern multimodal model training with a focus on diffusion models and multimodal large language models (MLLMs). We summarized all techniques into four categories: data quality, data quantity, data distribution and data safety. We further present our findings in the choice of data process methods in different type of models. This study aims to provide guidance to multimodal models developers with effective data processing techniques.