Part 09 · Fine-tuning and Serving Models
Project: Serve and Benchmark a Quantized Model
QuantizationServingLoad testingLatency & cost
Serve the same model at full precision and quantized, load-test both, and write down latency, throughput, memory and quality, then choose one with numbers.
⏱ 4-5 hours · Station 77 of 92