15. Pretraining at scale for many tasks
Foundation models are trained broadly and then adapted to many tasks. You will study pretraining, scale laws, emergent abilities, data mixtures, compute budgets, and the tradeoffs between closed and open models.