Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model
This research provides a comprehensive comparison of extreme 2-bit quantization methods for Polish large language models, offering insights into their performance and efficiency for researchers and developers working with resource-constrained NLP applications in less-resourced languages. It is an incremental contribution to the field of LLM quantization.
This study systematically evaluates six state-of-the-art 2-bit quantization methods on an 11B Polish language model, Bielik-11B-v2.3-Instruct. The best performing variant, QuIP# E8P12, achieved 71.92% across 22 Polish benchmarks, statistically matching the IQ2_XXS baseline (72.07%), and showed a 3.6 percentage point improvement on the eq_bench (47.14 vs 43.53).
We present Bielik-Q2-Sharp, the first systematic academic evaluation of extreme 2-bit quantization applied to a Polish large language model. Using Bielik-11B-v2.3-Instruct (11B parameters, Mistral architecture) as our base model, we compare six state-of-the-art post-training quantization methods -- QuIP#, SpinQuant+GPTQ, ButterflyQuant, QTIP, VPTQ, and AQLM -- all calibrated on a Polish-language corpus (CulturaX-PL) with shared Hessian matrices. Our best variant (QuIP# E8P12) achieves 71.92% across 22 Polish benchmarks versus 72.07% for the IQ2_XXS baseline -- within statistical noise, at a modest size premium (3.26 GB vs. ~2.6 GB). On eq_bench, our method scores 47.14 versus 43.53 (+3.6pp), suggesting superior preservation of higher-order reasoning. QTIP achieves the best per-bit efficiency (79.4% MC acc_norm at ~2.4 bpw, 3.27 GB), matching VPTQ's quality at 35% smaller size. We additionally document a MC-generation dissociation phenomenon where rotation-based methods preserve log-likelihood quality but fail catastrophically at autoregressive generation. The entire project was conducted by a single independent researcher on cloud GPUs (vast.ai) within a $285 budget. All models, Hessians, and evaluation logs are publicly available.