DatasetAgent: A Novel Multi-Agent System for Auto-Constructing Datasets from Real-World Images
This addresses the inefficiency of manual dataset creation for computer vision researchers, though it appears incremental as it builds on existing multi-agent and MLLM techniques.
The paper tackles the problem of manual dataset construction by proposing DatasetAgent, a multi-agent system that automatically builds image datasets from real-world images, achieving high-quality results used to train vision models for tasks like classification and detection.
Common knowledge indicates that the process of constructing image datasets usually depends on the time-intensive and inefficient method of manual collection and annotation. Large models offer a solution via data generation. Nonetheless, real-world data are obviously more valuable comparing to artificially intelligence generated data, particularly in constructing image datasets. For this reason, we propose a novel method for auto-constructing datasets from real-world images by a multiagent collaborative system, named as DatasetAgent. By coordinating four different agents equipped with Multi-modal Large Language Models (MLLMs), as well as a tool package for image optimization, DatasetAgent is able to construct high-quality image datasets according to user-specified requirements. In particular, two types of experiments are conducted, including expanding existing datasets and creating new ones from scratch, on a variety of open-source datasets. In both cases, multiple image datasets constructed by DatasetAgent are used to train various vision models for image classification, object detection, and image segmentation.