BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
This work addresses the need for bilingual medical AI tools, particularly for Arabic and English, by providing a model that supports diverse medical modalities, though it is incremental as it builds on existing multimodal and language model approaches.
The researchers tackled the challenge of creating a bilingual medical AI model by introducing BiMediX2, which achieved state-of-the-art results, including over 9% improvement in English and more than 20% in Arabic evaluations on their benchmark, and outperformed GPT-4 by approximately 9% in factual accuracy.
We introduce BiMediX2, a bilingual (Arabic-English) Bio-Medical EXpert Large Multimodal Model that supports text-based and image-based medical interactions. It enables multi-turn conversation in Arabic and English and supports diverse medical imaging modalities, including radiology, CT, and histology. To train BiMediX2, we curate BiMed-V, an extensive Arabic-English bilingual healthcare dataset consisting of 1.6M samples of diverse medical interactions. This dataset supports a range of medical Large Language Model (LLM) and Large Multimodal Model (LMM) tasks, including multi-turn medical conversations, report generation, and visual question answering (VQA). We also introduce BiMed-MBench, the first Arabic-English medical LMM evaluation benchmark, verified by medical experts. BiMediX2 demonstrates excellent performance across multiple medical LLM and LMM benchmarks, achieving state-of-the-art results compared to other open-sourced models. On BiMed-MBench, BiMediX2 outperforms existing methods by over 9% in English and more than 20% in Arabic evaluations. Additionally, it surpasses GPT-4 by approximately 9% in UPHILL factual accuracy evaluations and excels in various medical VQA, report generation, and report summarization tasks. Our trained models, instruction set, and source code are available at https://github.com/mbzuai-oryx/BiMediX2