Course page Home page Home page CoursesNew course

Language modeling and pretraining: Generative AI course | Zoonk

17. Language modeling and pretraining

Train causal and masked language models with next-token and denoising objectives. Cover pretraining corpora, context windows, decoding, perplexity, and emergent few-shot behavior.

Create this chapter

This chapter is part of the course, but it hasn't been created yet. Create it to see all of its lessons.

Create chapterNBack to courseEsc