34. Knowledge distillation and small language models

Transfer behavior from large teacher models into smaller models through logit, feature, and sequence-level distillation. Combine distillation with pruning and quantization to balance quality, speed, cost, and hardware limits.