Efficient Multimodal Learning from Data-centric Perspective
This work addresses the accessibility issue for broader research and user communities by reducing deployment barriers through efficient multimodal learning.
The paper tackles the high computational costs of Multimodal Large Language Models (MLLMs) by introducing Bunny, a family of lightweight MLLMs trained on high-quality data, which outperforms state-of-the-art large MLLMs on multiple benchmarks.
Multimodal Large Language Models (MLLMs) have demonstrated notable capabilities in general visual understanding and reasoning tasks. However, their deployment is hindered by substantial computational costs in both training and inference, limiting accessibility to the broader research and user communities. A straightforward solution is to leverage smaller pre-trained vision and language models, which inevitably cause significant performance drops. In this paper, we demonstrate the possibility of training a smaller but better MLLM with high-quality training data. Specifically, we introduce Bunny, a family of lightweight MLLMs with flexible vision and language backbones for efficient multimodal learning from selected training data. Experiments show that our Bunny-4B/8B outperforms the state-of-the-art large MLLMs on multiple benchmarks. We expect that this work can provide the community with a clean and flexible open-source tool for further research and development. The code, models, and data can be found in https://github.com/BAAI-DCAI/Bunny.