Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models
It addresses the need for better open-source chat models, particularly for Chinese language applications, but is incremental as it builds on existing instruction-tuning methods.
This project tackled the problem of improving open-source large language models by studying how training data factors like quantity, quality, and linguistic distribution affect performance in instruction-tuning, with a focus on English and Chinese, and provided publicly available models, data, and code for further development.
This project focuses on enhancing open-source large language models through instruction-tuning and providing comprehensive evaluations of their performance. We explore how various training data factors, such as quantity, quality, and linguistic distribution, influence the performance of instruction-tuned models trained on publicly accessible high-quality instruction datasets for both English and Chinese languages. Our goal is to supplement evaluation with quantitative analyses, providing valuable insights for the continued advancement of open-source chat models. Our model, data, and code are publicly available for others to use and build upon.