Centre the data, or the code spends itself on the mean

Without centring, the matrix in (I.12.2) is the second-moment matrix $\frac1n\mat X^{\mathsf T}\mat X$ and not the covariance. Apply I.12.T1 to whichever matrix the loss contains.

As they are. The second-moment matrix is

The algebra of I.12.T1 applies to this matrix. Its largest eigenvalue is $100$, with eigenvector $(1,0)$, so the autoencoder keeps the first coordinate. All three codes are $10$. Every input is reconstructed as $(10,0)$ and the loss is the other eigenvalue, $2/3=0.666667$.

The code is the same number for every example. It carries no information about which example it came from.

Centred. The mean is $(10,0)$, and the centred inputs are $(0,-1)$, $(0,0)$ and $(0,1)$. The covariance is $\begin{bmatrix}0&0\\0&2/3\end{bmatrix}$. Its largest eigenvalue is $2/3$, with eigenvector $(0,1)$. The codes are $-1$, $0$ and $1$, and the loss is the other eigenvalue, $0$.

The second result is the principal subspace: the direction along which the data vary. The first is the direction in which the data are large. The two matrices differ by $\vec\mu\vec\mu^{\mathsf T}$ for the mean $\vec\mu$, and when the mean is long compared with the spread, that term dominates and the leading direction is close to $\vec\mu$ itself.

An autoencoder with a bias in the decoder can learn the mean and escape this. Without one, centre the data first.