Introducing DictaLM -- A Large Generative Language Model for Modern Hebrew
This provides a foundational model for Hebrew-specific NLP tasks, though it is incremental as it adapts existing LLM approaches to a new language.
The authors tackled the lack of large language models for Modern Hebrew by introducing DictaLM, a 7B-parameter model trained on Hebrew-centric data, and released it under a Creative Commons license to support the Hebrew NLP community.
We present DictaLM, a large-scale language model tailored for Modern Hebrew. Boasting 7B parameters, this model is predominantly trained on Hebrew-centric data. As a commitment to promoting research and development in the Hebrew language, we release both the foundation model and the instruct-tuned model under a Creative Commons license. Concurrently, we introduce DictaLM-Rab, another foundation model geared towards Rabbinic/Historical Hebrew. These foundation models serve as ideal starting points for fine-tuning various Hebrew-specific tasks, such as instruction, Q&A, sentiment analysis, and more. This release represents a preliminary step, offering an initial Hebrew LLM model for the Hebrew NLP community to experiment with.