Self-Pluralising Culture Alignment for Large Language Models
This addresses the challenge of making LLMs serve diverse cultural values, which is crucial as models become globally accessible, though it appears incremental as it builds on existing fine-tuning and prompt engineering techniques.
The paper tackles the problem of aligning large language models (LLMs) to pluralistic human values across cultures by proposing CultureSPA, a framework that detects and fine-tunes models using culture-related instances, resulting in significant improvements in cultural alignment without compromising general abilities, with further gains possible through prompt engineering.
As large language models (LLMs) become increasingly accessible in many countries, it is essential to align them to serve pluralistic human values across cultures. However, pluralistic culture alignment in LLMs remain an open problem. In this paper, we propose CultureSPA, a Self-Pluralising Culture Alignment framework that allows LLMs to simultaneously align to pluralistic cultures. The framework first generates questions on various culture topics, then yields LLM outputs in response to these generated questions under both culture-aware and culture-unaware settings. By comparing culture-aware/unaware outputs, we are able to detect and collect culture-related instances. These instances are employed to fine-tune LLMs to serve pluralistic cultures in either a culture-joint or culture-specific way. Extensive experiments demonstrate that CultureSPA significantly improves the alignment of LLMs to diverse cultures without compromising general abilities. And further improvements can be achieved if CultureSPA is combined with advanced prompt engineering techniques. Comparisons between culture-joint and culture-specific tuning strategies, along with variations in data quality and quantity, illustrate the robustness of our method. We also explore the mechanisms underlying CultureSPA and the relations between different cultures it reflects.