Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models
This work addresses the need for advanced AI models tailored to the Thai language and culture, offering multimodal capabilities for tasks like document understanding and speech processing, but it is incremental as it builds on existing open models like Llama 3 and Qwen 2.
The paper introduces Typhoon 2, a family of open text and multimodal large language models optimized for Thai, including models for text, vision, and audio, with sizes ranging from 1 to 70 billion parameters and enhanced Thai performance while preserving base capabilities.
This paper introduces Typhoon 2, a series of text and multimodal large language models optimized for the Thai language. The series includes models for text, vision, and audio. Typhoon2-Text builds on state-of-the-art open models, such as Llama 3 and Qwen2, and we perform continual pre-training on a mixture of English and Thai data. We employ post-training techniques to enhance Thai language performance while preserving the base models' original capabilities. We release text models across a range of sizes, from 1 to 70 billion parameters, available in both base and instruction-tuned variants. To guardrail text generation, we release Typhoon2-Safety, a classifier enhanced for Thai cultures and language. Typhoon2-Vision improves Thai document understanding while retaining general visual capabilities, such as image captioning. Typhoon2-Audio introduces an end-to-end speech-to-speech model architecture capable of processing audio, speech, and text inputs and generating both text and speech outputs.