Chapter VII
The Transformer
The block, assembled.
What this chapter covers
- Transformer block
- Multi-head attention
- Feed-forward network
- Residual connections
- Layer normalisation
- Encoder
- Decoder
- Encoder-decoder models
- Causal masking
- Full forward pass
Propositions
- Prop. 1A transformer block mixes across positions, then computes within one.Attention is the only part of a block that moves information between tokens. The feed-forward network sees each position alone, and the two alternate for the whole depth of the model.
- Prop. 2Heads are slices of one budget, not copies of one mechanism.Multi-head attention splits the width into parts and runs attention in each. It costs what single-head attention costs, and buys several relations at once at the price of resolution inside each.
- Prop. 3Layers write into a shared stream rather than replacing it.Every sublayer adds its output to the running representation. This is why gradients reach the first layer, and why a state from partway up the stack can be read with the output head.
- Prop. 4Where the normalisation sits decides whether the shortcut is clean.Post-norm places a normalisation on the residual path and needs a warm-up to train deeply. Pre-norm leaves the path untouched and trains from a flat start. Same components, different order.
- Prop. 5A causal mask is what turns a transformer into a language model.Setting the future to minus infinity before the softmax lets every position in a sequence be trained as its own prediction problem, in a single forward pass.
- Prop. 6Encoder, decoder and encoder–decoder differ only in what each stack may look at.The block is unchanged in all three. What differs is the mask, and whether a second stack supplies the keys and values.