Pooling removes information within a window, but does not guarantee shift invariance.

A reduction needs a window and a stride

For $m$ entries in one window,

Equation (I.11.7) says nothing yet about how windows are placed. A $2$ by $2$ pooling window at stride two gives disjoint windows; stride one gives overlapping windows. Pooling usually operates independently per channel, so it need not change the channel count.

For an unpadded window with a unique maximum at index $r$, the derivative of max pooling is one at $r$ and zero elsewhere. An upstream scalar $g$ therefore returns $g$ only to that entry. Average pooling returns $g/m$ to each entry. Overlapping windows add their backward contributions just as convolution does.

A tie makes the maximum non-differentiable in the ordinary sense. An implementation may choose one maximising index; more generally, convex combinations of their one-hot gradients are valid subgradients. A central finite difference at a tie is not a unique analytic-gradient oracle. I.11.X05 separates these cases.

At padded boundaries, state the rule. Ignoring padded entries in an average uses a different divisor from counting them as zeros. For max pooling, zero padding can even exceed negative real entries; an implementation using negative infinity has different boundary behaviour.

Moving inside a window is not moving across the grid

A window containing $(1,0)$ and one containing $(0,1)$ both have maximum one. The pooled scalar cannot recover which entry was nonzero. This is a real loss of location within that window.

But a one-position shift can move the response into another window. Stride-two max pooling maps $(0,1,0,0)$ to $(1,0)$ and its right shift $(0,0,1,0)$ to $(0,1)$. The output is not invariant. The diagram shows both the within-window equality and this boundary crossing. I.11.B04 also compares means.

Subsampling can alias fine patterns: different sampling phases may return very different values. Filtering before subsampling can reduce this sensitivity, as studied by Zhang. It does not create an exact invariance guarantee for arbitrary finite images and every boundary rule.

Global pooling changes the question the head can answer

Global average pooling returns one number per channel by averaging all its spatial entries. It is invariant to a permutation of those entries at the feature-map level. That does not establish invariance of the entire image classifier to every spatial transformation: the preceding feature extractor and image boundaries still matter.

At $64$ channels and $7$ by $7$ positions, a flattened ten-class affine head has $31,370$ parameters. Pooling first reduces the head to $650$. I.11.X10 derives both counts and identifies the information the smaller head can no longer use.

For segmentation, a decoder needs spatially organised evidence. Upsampling increases the output grid’s size but cannot uniquely undo a many-to-one pooling map. High-resolution skip features can carry information that the coarse representation discarded. They are a useful design choice, not a proof that every decoder must reconstruct every original pixel.

A strided convolution is learned rather than fixed pooling, but its information loss depends on the whole mapping and channel count. It is too broad to say that every spatial downsampling operation must lose information: space-to-depth rearrangement, for example, reduces height and width while preserving every value in extra channels. Count the representation and inspect the map, not just the spatial dimensions.