Mixture-of-experts models: Generative AI course | Zoonk
19. Mixture-of-experts models
Use sparse expert routing to increase model capacity without activating every parameter for every token. Examine load balancing, routing stability, communication costs, and mixture-of-experts serving.
