LLM quantization

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

13 critique · 28 beaten on benchmarks

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

9 critique · 15 beaten on benchmarks

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

13 critique · 9 beaten on benchmarks

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs

7 critique · 12 beaten on benchmarks

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models

2 critique · 14 beaten on benchmarks

2 critique · 10 beaten on benchmarks

3 critique · 8 beaten on benchmarks

3 critique · 5 beaten on benchmarks

QuIP: 2-Bit Quantization of Large Language Models With Guarantees

3 critique · 4 beaten on benchmarks

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

3 critique · 4 beaten on benchmarks

PB-LLM: Partially Binarized Large Language Models

4 critique · 3 beaten on benchmarks

FlatQuant: Flatness Matters for LLM Quantization

2 critique · 4 beaten on benchmarks