Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languages
This addresses the problem of inadequate representation of Indian languages in multimodal AI for researchers and developers, though it is incremental as it extends existing data collection methods to new languages.
The authors tackled the lack of multimodal datasets for Indian languages by introducing the Chitrakshara dataset series, which includes 193M images and 30B text tokens across 11 languages, aiming to support culturally inclusive Vision-Language Models.
Multimodal research has predominantly focused on single-image reasoning, with limited exploration of multi-image scenarios. Recent models have sought to enhance multi-image understanding through large-scale pretraining on interleaved image-text datasets. However, most Vision-Language Models (VLMs) are trained primarily on English datasets, leading to inadequate representation of Indian languages. To address this gap, we introduce the Chitrakshara dataset series, covering 11 Indian languages sourced from Common Crawl. It comprises (1) Chitrakshara-IL, a large-scale interleaved pretraining dataset with 193M images, 30B text tokens, and 50M multilingual documents, and (2) Chitrakshara-Cap, which includes 44M image-text pairs with 733M tokens. This paper details the data collection pipeline, including curation, filtering, and processing methodologies. Additionally, we present a comprehensive quality and diversity analysis to assess the dataset's representativeness across Indic languages and its potential for developing more culturally inclusive VLMs.