AnswerCarefully: A Dataset for Improving the Safety of Japanese LLM Output
This addresses safety concerns for Japanese LLM users by providing a culturally relevant dataset, though it is incremental as it adapts existing risk categories to a new language.
The authors tackled the problem of improving the safety of Japanese LLM outputs by creating AnswerCarefully, a dataset of 1,800 question-answer pairs tailored to Japan's socio-cultural context, which when used for fine-tuning improved safety without compromising general utility.
In this paper we present AnswerCarefully, a dataset for promoting the safety and appropriateness of Japanese LLM outputs. The dataset consists of 1,800 pairs of questions and reference answers, where the questions require special attention in answering. It covers a wide range of risk categories established in prior English-language datasets, but the data samples are original in that they are manually created to reflect the socio-cultural context of LLM usage in Japan. We show that using this dataset for instruction to fine-tune a Japanese LLM led to improved output safety without compromising the utility of general responses. We also report the results of a safety evaluation of 12 Japanese LLMs using this dataset as a benchmark. Finally, we describe the latest update on the dataset which provides English translations and annotations of the questions, aimed at facilitating the derivation of similar datasets in different languages and regions.