Enhancing Japanese Large Language Models with Reasoning Vectors
This work addresses the problem of resource-efficient enhancement for Japanese LLMs, with potential implications for other languages, though it appears incremental in approach.
The authors tackled the challenge of improving Japanese large language models (LLMs) with limited resources by applying reasoning vectors extracted from reasoning LLMs, resulting in a simple and effective method that boosts performance.
Post-training methods have improved the performance and enhanced the reasoning capability for mainstream large language models (LLMs), but the same is challenging for Japanese LLMs to achieve due to the amount of resources required. Inspired by task vectors that extract the change of weights before and after training, specifically for a certain task, we obtain reasoning vectors from reasoning LLMs and apply them to Japanese LLMs to boost their performance. While the resources available present a challenge to improve Japanese LLMs, we present a simple and effective way to obtain high improvement and hope to inspire for other languages.