Two estimators of one gradient

STATEMENT

Show that the pathwise and score-function estimators of one gradient are both unbiased, compute their variances, and follow them as the encoder’s distribution narrows.

GIVEN

$q=\mathcal N(\mu,s^2)$, $z=\mu+s\varepsilon$ with $\varepsilon\sim\mathcal N(0,1)$, and $f(z)=z^2$. The target is $\partial\,\E_q[f(z)]/\partial\mu$. The moments $\E[\varepsilon^2]=1$, $\E[\varepsilon^4]=3$ and $\E[\varepsilon^6]=15$, and odd moments vanish.

FIND

Both one-draw estimates, their means, their variances in general and at $\mu=1$, $s=0.5$, and their behaviour as $s\to0$.

STRATEGY

Write each estimate as a polynomial in $\varepsilon$ and take moments term by term.

SOLUTION

The target. $\E_q[z^2]=\mu^2+s^2$, so the derivative is $2\mu$.

Pathwise. By (I.13.6) the estimate is $g_{\mathrm p}=f'(z)=2(\mu+s\varepsilon)$. Its mean is $2\mu$ and its variance is

Score function. By (I.13.7), with $\partial\log q/\partial\mu=(z-\mu)/s^2=\varepsilon/s$,

Expanding, $(\mu+s\varepsilon)^2\varepsilon =\mu^2\varepsilon+2\mu s\varepsilon^2+s^2\varepsilon^3$, whose mean is $2\mu s$. So $\E g_{\mathrm s}=2\mu$: unbiased. For the second moment,

which averages to $\mu^4+18\mu^2s^2+15s^4$. Dividing by $s^2$ and subtracting the squared mean,

The numbers. At $\mu=1$, $s=0.5$ the pathwise variance is $1$ and the score-function variance is $(1+4.5+0.9375)/0.25-4=21.75$.

As $s\to0$. The pathwise variance $4s^2$ goes to zero. The score-function variance behaves like $\mu^4/s^2$ and grows without bound:

Check — sanity

At $\mu=0$ the score-function variance is $15s^2$, finite and small, while the pathwise variance is $4s^2$. The score function is worst when $f$ is large where $q$ sits, because it multiplies $f$ itself by $\varepsilon/s$.