Count what the classifier keeps
complexity▲▲△A CNN returns 64 feature channels on a by grid. Compare a flattened affine head to ten classes with global average pooling followed by an affine ten-class head. Include biases. What distinction can the flattened head express that the pooled head cannot?
Hint
Flattening preserves 49 coordinates per channel; averaging retains one.
Solution
Flattening gives inputs. The affine head has parameters. After global averaging it has .
A flattened head can weight a feature differently at two positions. A pooled head receives the same channel average after any permutation within that channel and cannot distinguish those permutations from that representation alone.
The parameter saving does not prove that the smaller head is better. Location may be nuisance variation for one task and essential evidence for another. The upstream feature extractor can also encode some positional information before pooling, so the limitation is stated at the feature-map interface, not as a universal claim about all CNN classifiers.