January Food Benchmark (JFB): A Public Benchmark Dataset and Evaluation Suite for Multimodal Food Analysis
This provides a standardized evaluation dataset and framework for the research community working on multimodal food analysis, addressing a critical bottleneck in the field.
The authors tackled the lack of standardized evaluation in automated nutritional analysis by introducing the January Food Benchmark (JFB), a public dataset of 1,000 food images with human-validated annotations, and a benchmarking framework, resulting in a specialized model achieving an Overall Score of 86.2, a 12.1-point improvement over general-purpose models.
Progress in AI for automated nutritional analysis is critically hampered by the lack of standardized evaluation methodologies and high-quality, real-world benchmark datasets. To address this, we introduce three primary contributions. First, we present the January Food Benchmark (JFB), a publicly available collection of 1,000 food images with human-validated annotations. Second, we detail a comprehensive benchmarking framework, including robust metrics and a novel, application-oriented overall score designed to assess model performance holistically. Third, we provide baseline results from both general-purpose Vision-Language Models (VLMs) and our own specialized model, january/food-vision-v1. Our evaluation demonstrates that the specialized model achieves an Overall Score of 86.2, a 12.1-point improvement over the best-performing general-purpose configuration. This work offers the research community a valuable new evaluation dataset and a rigorous framework to guide and benchmark future developments in automated nutritional analysis.