Tokenizers and model inputs

Trace how a tokenizer turns text into token IDs and decodes IDs back into text. Inspect special tokens, padding, truncation, and attention masks that prepare variable-length text for a model.