The finite logit gap label smoothing asks for

The gradient is still $\vec{p} - \vec{y}'$ by I.4.B03 — that derivation never needed $\vec{y}$ to be one-hot, only to sum to one.

Step 1 — the smoothed target. For true class $c$,

Check it sums to one: $(1-\varepsilon)(1) + K\cdot\varepsilon/K = 1 - \varepsilon + \varepsilon = 1$ ✓. That is the only property I.4.B03’s Step 8 used, so the gradient result carries over unchanged:

$\frac{\partial L}{\partial \vec{z}} = \vec{p} - \vec{y}'$

Step 2 — set the gradient to zero. The stationary point is $\vec{p} = \vec{y}'$:

These are attainable probabilities, both strictly inside $(0,1)$. Contrast with $\varepsilon = 0$, where the requirement is $p_c = 1$ and $p_j = 0$ — values softmax approaches but never reaches for finite logits.

Step 3 — convert to a logit gap. From $p_i = e^{z_i}/S$, the ratio of two probabilities has the shared denominator cancel:

$\frac{p_c}{p_j} = \frac{e^{z_c}}{e^{z_j}} = e^{\,z_c - z_j}$

Taking logarithms,

Step 4 — simplify. Multiply numerator and denominator by $K$:

Step 5 — evaluate at $\varepsilon = 0.1$, $K = 3$.

Or from the closed form: $\big(3(0.9) + 0.1\big)/0.1 = 2.8/0.1 = 28$ ✓.

Step 6 — the limits. As $\varepsilon \to 0^{+}$ the argument of the logarithm is $K/\varepsilon \to \infty$, so the gap diverges — recovering the unsmoothed case where no finite logit configuration is optimal. As $\varepsilon \to 1$ the argument tends to $1$ and the gap to $0$: the target becomes uniform and the model is asked to predict nothing at all.

What this buys, and what it costs.

Bounded logits. The optimum is at a gap of $3.33$ rather than at infinity, so there is no incentive to keep growing the weights. That is a regularisation effect, obtained without a penalty term.

Better calibration. An unsmoothed model driven toward $p_c = 1$ is systematically overconfident. Smoothing caps the confidence at a chosen value, and $0.9333$ is a defensible one.

Worse for distillation. Müller et al. (2019) show smoothing collapses the geometry of the penultimate layer: the logits of the wrong classes are pushed to be equally wrong, destroying the relative information a student model would learn from. A smoothed teacher is a worse teacher, and this is the standard reason not to smooth when distillation is planned.

A caution on reading the loss. The minimum value is no longer zero. At the optimum the loss equals the entropy of $\vec{y}'$, which for $\varepsilon = 0.1$, $K = 3$ is $-0.9333\log 0.9333 - 2(0.0333)\log 0.0333 = 0.0644 + 0.2266 = 0.2910$ nats. A smoothed run that plateaus at $0.29$ has converged; comparing it to an unsmoothed run’s $0.05$ is comparing two different objectives.