Part 09 · Fine-tuning and Serving Models
Quantization and Distillation
INT8 & INT4GPTQ & AWQDistillationQuality loss
Making models smaller: storing weights in fewer bits, training a small student from a big teacher, and measuring how much quality you gave up.
⏱ 1-2 hours · Station 73 of 92