Model compression
10 methods in the atlas attack this one problem. They are rivals: each wins something the others do not.
Phrasings that mean this problem
Model compressionModel quantizationExtreme quantization
deep-learning
- Knowledge distillationSoft-label teacher matchingstandarddeep-learning
- Network pruningMagnitude-based sparsificationstandarddeep-learning
- Quantization-aware trainingstandalonespecialistdeep-learning
llm-inference
- GPTQSecond-order layerwise roundingstandardllm-inference
- AWQActivation-aware channel scalingstandardllm-inference
- SmoothQuantActivation-weight scale migrationspecialistllm-inference
- LLM.int8()Mixed-precision outlier decompositionspecialistllm-inference
- K-quant block quantizationMixed bit widths per blockspecialistllm-inference
- BitNetTernary weight quantizationspecialistllm-inference
llm-training-alignment
- Sequence-level distillationTeacher-generated targetsspecialistllm-training-alignment