Md. Asif Uddin
    Problem I.11.B01

    Slide one kernel, keep its orientation

    numeric▲△△

    Exact integers and fractions where possible; six decimals otherwise.

    STATEMENT

    Compute every output of a small valid cross-correlation.

    GIVEN

    One input channel and one output channel, stride one, no padding, dilation one and zero bias:

    X=[120013210],K=[10−12].\begin{aligned} X&=\begin{bmatrix}1&2&0\\0&1&3\\2&1&0\end{bmatrix},\\[6pt] K&=\begin{bmatrix}1&0\\-1&2\end{bmatrix}. \end{aligned}

    FIND

    The output shape, four output values, parameter count with a trainable bias, and main-product multiply-accumulates.

    STRATEGY

    Place the stored kernel on each complete window. Do not flip it. Use (I.11.4) for shape and (I.11.2) for the count.

    SOLUTION

    Each axis has 3−2+1=23-2+1=2 outputs. The four sums are

    Y0,0=1(1)+2(0)+0(−1)+1(2)=3,Y0,1=2(1)+0(0)+1(−1)+3(2)=7,Y1,0=0(1)+1(0)+2(−1)+1(2)=0,Y1,1=1(1)+3(0)+1(−1)+0(2)=0.\begin{aligned} Y_{0,0}&=1(1)+2(0)\\ &\quad+0(-1)+1(2)=3,\\ Y_{0,1}&=2(1)+0(0)\\ &\quad+1(-1)+3(2)=7,\\ Y_{1,0}&=0(1)+1(0)\\ &\quad+2(-1)+1(2)=0,\\ Y_{1,1}&=1(1)+3(0)\\ &\quad+1(-1)+0(2)=0. \end{aligned}

    Thus Y=[3700]Y=\begin{bmatrix}3&7\\0&0\end{bmatrix}. There are four kernel weights and one bias, or five parameters. Four output windows use four weights each, giving 16 multiply-accumulates. A zero current bias is still a parameter if it is trainable.

    Answer

    Output shape (1,2,2)(1,2,2), values [[3,7],[0,0]][[3,7],[0,0]], five parameters and 16 main-product multiply-accumulates.

    Check — sanity

    The lower windows contain nonzero inputs but have zero output because positive and negative contributions cancel. A zero response need not mean an empty patch.

    Check — numeric · i-11-convolution-checks.py

    The reproduction script uses only Python’s standard library. From a repository checkout, run:

    python3 scripts/snippets/i-11-convolution-checks.py

    It checks the forward values, every input and kernel derivative, and all worked-problem and exercise calculations.

    Executed in CI. The digits above are the digits it printed.

    Where this breaks

    Flipping the kernel changes the operation. Adding channels requires summing their responses before adding the output channel’s bias once.

    Variation

    Set the bias to one. Every output increases by one, producing [[4,8],[1,1]][[4,8],[1,1]], while the parameter count remains five.

    Draws on