Culinary Class Wars: Evaluating LLMs using ASH in Cuisine Transfer Task
This addresses the challenge of cultural accuracy in AI-generated recipes for culinary applications, but it is incremental as it builds on existing LLM evaluation methods.
The study tackled the problem of LLMs struggling with culinary creativity in adapting recipes to cultural requirements by introducing the ASH benchmark to evaluate their performance in cuisine transfer tasks, revealing insights into their generative and evaluative capabilities.
The advent of Large Language Models (LLMs) have shown promise in various creative domains, including culinary arts. However, many LLMs still struggle to deliver the desired level of culinary creativity, especially when tasked with adapting recipes to meet specific cultural requirements. This study focuses on cuisine transfer-applying elements of one cuisine to another-to assess LLMs' culinary creativity. We employ a diverse set of LLMs to generate and evaluate culturally adapted recipes, comparing their evaluations against LLM and human judgments. We introduce the ASH (authenticity, sensitivity, harmony) benchmark to evaluate LLMs' recipe generation abilities in the cuisine transfer task, assessing their cultural accuracy and creativity in the culinary domain. Our findings reveal crucial insights into both generative and evaluative capabilities of LLMs in the culinary domain, highlighting strengths and limitations in understanding and applying cultural nuances in recipe creation. The code and dataset used in this project will be openly available in \url{http://github.com/dmis-lab/CulinaryASH}.