Chapter II
Autoregressive Transformers
Decoder-only models, and what makes them generate.
What this chapter covers
- Causal masking
- Decoder-only architecture
- GPT-style models
- Context windows
- KV cache
Propositions
Not yet written. The topics above are the plan for this chapter; each will become a proposition with its own figure.