A recurrent network carries a fixed-width state through a sequence. Learning what to remember requires differentiating every use of the shared weights.
Shared state, backpropagation through time, gradient transport, and the cost of remembering a sequence.
How this chapter is built
M3Load-bearing
The content is mathematics. Understanding is demonstrated by computation, not recall.
basics11/11what the words mean
concept2/2what to picture
theory4/4why it works, and when it does not
mathematics15/15derive it, then compute it
practice9/9build it, break it, read the papers
Five strands, not one. Mathematics is the spine; the other four are the body. A chapter cannot pay its way out of teaching with problems, nor out of problems with teaching.
A dense layer over a flattened sequence assigns different weights to different positions and requires a chosen input length. A recurrent cell reuses one transition at every position. That buys variable-length processing and a compact streaming state, but introduces a long dependency chain. We build that chain, differentiate it and test where the backward signal is lost.
Fig. 9 — A recurrence unrolled. Every step passes a fixed-width state to the next, so the path between two distant positions is as long as the distance between them.
What this chapter covers
One transition, shared across time
Sequence outputs, final-state outputs and causal prediction
Unrolling the graph without copying the parameters
Backpropagation through time and summed parameter gradients
Products of state-dependent Jacobians
Vanishing gradients, saturation and transient amplification
Gradient norm clipping and truncated backpropagation
Padding masks, state resets and sequence boundaries
Parameter counts, training memory and streaming inference
Dilated convolutions and the path to attention
Apparatus
The mathematics this chapter leans on, held in Book 0 so it can be assumed here without being taught here. Not a gate — follow a link when a step stops making sense.
λA regularisation coefficient, or an eigenvalue where the context is linear algebra
Definitions
Definition 1Sequence
An ordered list of inputs x_1 through x_T. Position affects meaning, and examples may have different lengths. The input width D is fixed even when T varies.
Definition 2Hidden state
A vector h_t in R^H carried from one time step to the next, computed from the preceding state and current input. Its width H does not grow with the number of inputs read.
Definition 3Parameter sharing
Reusing the same trainable matrices and biases at every time step. Unrolling creates multiple uses of each parameter, not independent copies. Their gradient contributions must be added.
Definition 4Unrolling
Drawing a finite recurrence as an acyclic computation graph with one cell application per time step. Ordinary reverse-mode differentiation on this graph is backpropagation through time.
Definition 5Backpropagation through time
The backward sweep through the unrolled graph. At each state, the gradient combines the local output loss with the signal returned from later states, then differentiates the cell and accumulates shared-parameter gradients.
Definition 6Temporal Jacobian
The derivative of h_t with respect to h_(t-1), holding input and parameters fixed. For a tanh cell it is diag(1 - h_t²) W, so shared W still gives different Jacobians along a trajectory.
Definition 7Vanishing or exploding gradient
Attenuation or amplification of a backward signal through many temporal Jacobians. This describes a derivative, not necessarily the magnitude of the hidden states. Bounded tanh states can coexist with large gradients.
Definition 8Truncated backpropagation through time
Carrying a state's value into the next chunk while stopping its derivative at the boundary. This reduces the retained graph and omits cross-boundary credit assignment. Detaching is different from resetting to zero.
Definition 9Global norm clipping
Rescaling the concatenated parameter-gradient vector when its Euclidean norm exceeds a positive threshold. Its direction is preserved. Clipping coordinates or tensors separately is a different operation.
Definition 10Padding mask
An indicator distinguishing real positions from padding. It can exclude padded losses and stop padded steps from changing the state. A zero input alone does neither.
Definition 11Causal computation
An output at time t that depends only on inputs available through t. For next-token prediction the target is x_(t+1). A bidirectional encoder reads later inputs and is not a causal next-token predictor.
Formal results
I.9.T1
Shared weights receive the sum of their temporal contributions
For a_t = Ux_t + Wh_(t-1) + b, h_t = tanh(a_t), z_t = Vh_t + c, and a sum of per-step losses, let e_t = d ell_t/d z_t. Then delta_t = (Vᵀ e_t + Wᵀ delta_(t+1)) elementwise-multiplied by (1 - h_t²), with delta_(T+1) = 0. The recurrent-weight gradient is the sum of delta_t h_(t-1)ᵀ over all t. I.9.B02 derives the rule.
What this does not promise
This assumes the full unrolled graph, fixed inputs, and no extra parameter dependence in the loss. Truncation drops paths, regularisation adds its own derivative, and a learned initial state receives a gradient too.
I.9.T2
Uniform contraction bounds distant sensitivity
Writing A_t = d h_t/d h_(t-1), the derivative d h_T/d h_s is A_T A_(T-1) ... A_(s+1). Its spectral norm is bounded by the product of the factor norms. If each is at most r < 1, the bound is r^(T-s).
What this does not promise
A large upper bound does not prove growth. Factors may contract different directions, and the incoming gradient may miss an amplified direction. Eigenvalues of W alone do not determine finite-horizon behaviour; I.9.B03 and I.9.X05 give counterexamples.
I.9.T3
Finite-precision state has finite distinguishable histories
If an H-coordinate state has at most 2^p representable values per coordinate, it has at most 2^(pH) states. There are 2^T binary histories of length T. If T > pH, at least two histories map to the same state. A future computation given only that state cannot distinguish them.
What this does not promise
This finite-state counting argument does not say every task needs the entire history. Parity needs only two states. With ideal infinite-precision reals the finite-state premise no longer holds.
Assumptions
Each is paired with the problem that shows what its removal costs. An assumption nobody tests is a disclaimer.
State is carried only between chunks of the same stream, with batch rows preserving stream identity. Independent examples receive independent initial states.I.9.X06
A local Jacobian bound holds along the actual trajectory. A large norm or an eigenvalue calculation alone does not establish exploding gradients.I.9.X05
A causal predictor cannot access inputs later than the prediction time.I.9.X09
A fixed-width storage argument assumes finitely many representable states, not arbitrary-precision real encoding.I.9.X08
Assigning rather than accumulating a shared-weight gradient keeps only one time step. In I.9.B02 the recurrent contributions are 0, 1.5 and 7.5; keeping only the last reports 7.5 instead of 9.
Masking the loss but reading the state after padding changes a representation. For h_next = 0.5 h + x, state 2 becomes 1 after zero padding. Freeze the padded state, pack the sequence, or gather the last valid output.
Carrying state between shuffled, unrelated examples leaks one example into another. Detaching prevents a gradient crossing the boundary but leaves the leaked value unchanged.
Clipping a finite gradient limits its norm; it does not restore vanished components or undo NaNs already created. Check finiteness before relying on clipping.
One-step truncation in I.9.B05 keeps the full pass's prediction and loss while changing dL/dw from 9 to 7.5 and dL/du from 3.75 to zero. Forward equality is not a gradient test.