Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning
This work addresses the alignment of LLMs with cultural values, which is important for users in culturally diverse contexts, but it is incremental as it builds on existing in-context learning methods.
The paper tackled the problem of aligning Large Language Models with cultural values by using in-context learning and human survey data at inference time, resulting in improved alignment across 5 models, including English-centric and multilingual ones, and showing applicability to test languages beyond English and diverse countries.
Improving the alignment of Large Language Models (LLMs) with respect to the cultural values that they encode has become an increasingly important topic. In this work, we study whether we can exploit existing knowledge about cultural values at inference time to adjust model responses to cultural value probes. We present a simple and inexpensive method that uses a combination of in-context learning (ICL) and human survey data, and show that we can improve the alignment to cultural values across 5 models that include both English-centric and multilingual LLMs. Importantly, we show that our method could prove useful in test languages other than English and can improve alignment to the cultural values that correspond to a range of culturally diverse countries.