Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation
This work addresses the problem of evaluating MLLMs' capabilities in specialized computer vision tasks for researchers and practitioners, though it is incremental as it builds on existing models and benchmarks.
The study compared general-purpose multimodal large language models (MLLMs) like ShareGPT4V, ChatGPT, and LLaVA-Next against a specialized model, MiVOLO, for age and gender estimation, revealing insights into their strengths and weaknesses, and attempted fine-tuning ShareGPT4V to achieve state-of-the-art results in this task.
Multimodal Large Language Models (MLLMs) have recently gained immense popularity. Powerful commercial models like ChatGPT-4V and Gemini, as well as open-source ones such as LLaVA, are essentially general-purpose models and are applied to solve a wide variety of tasks, including those in computer vision. These neural networks possess such strong general knowledge and reasoning abilities that they have proven capable of working even on tasks for which they were not specifically trained. We compared the capabilities of the most powerful MLLMs to date: ShareGPT4V, ChatGPT, LLaVA-Next in a specialized task of age and gender estimation with our state-of-the-art specialized model, MiVOLO. We also updated MiVOLO and provide details and new metrics in this article. This comparison has yielded some interesting results and insights about the strengths and weaknesses of the participating models. Furthermore, we attempted various ways to fine-tune the ShareGPT4V model for this specific task, aiming to achieve state-of-the-art results in this particular challenge. Although such a model would not be practical in production, as it is incredibly expensive compared to a specialized model like MiVOLO, it could be very useful in some tasks, like data annotation.