A tied weight that receives half its gradient

$w$ has two consumers. (I.6.2) says its adjoint is a sum over them, and an assignment keeps whichever term arrives last.

Correct. Substituting, $v = w^2x$ and $L = \tfrac{1}{2}w^4x^2$, so

By the graph. $\bar{v} = v = w^2x$. The node $v = wu$ has two producers:

The node $u = wx$ then contributes

By (I.6.2) the adjoint of $w$ is the sum of the contributions from its consumers:

which matches the direct computation.

With assignment instead of accumulation. Whichever contribution is written last survives, and each is exactly $w^3x^2$ — half the truth. The gradient is not noisy, not biased in a complicated way, and not sometimes right: it is consistently a factor of two short.

Why that is worse than it sounds. A gradient scaled by a constant is still a descent direction, so training does not diverge or produce obvious garbage. It converges, more slowly, and looks like a learning-rate problem. Now make the two uses asymmetric — an embedding matrix that is also the output projection, a convolutional filter applied at 4096 positions, a recurrent matrix at 512 timesteps — and the surviving term is one of many, in a proportion that varies per example. That is not a rescaled gradient; it is the gradient of a different function, and it descends confidently to the wrong place.

Which is why frameworks accumulate. .grad is added into, never assigned, and the mirror-image bug follows immediately: forget to zero it between steps and every step’s gradient is the sum of all steps so far. Both errors come from the same line of (I.6.2), read in opposite directions.