Equivariance, proved and broken

$\mat P^{\mathsf T}\mat P=\mat I$, an entrywise function commutes with reordering rows, and $\mathbf 1^{\mathsf T}\mat P=\mathbf 1^{\mathsf T}$.

Equivariance. Relabel: $\mat A\mapsto\mat P\mat A\mat P^{\mathsf T}$ and $\mat H\mapsto\mat P\mat H$. The pre-activation becomes

An entrywise $\sigma$ applied to a matrix with reordered rows gives the same rows reordered, so the output is $\mat P\mat H'$. $\blacksquare$

Invariance. A permutation matrix’s columns each contain one $1$, so $\mathbf 1^{\mathsf T}\mat P=\mathbf 1^{\mathsf T}$ and $\mathbf 1^{\mathsf T}(\mat P\mat H')=\mathbf 1^{\mathsf T}\mat H'$. $\blacksquare$

A layer that breaks it. On the paw the lowest-numbered neighbours of nodes $1$ to $4$ are $2$, $1$, $1$ and $3$, so the layer returns $(2,1,1,3)$. After swapping nodes $1$ and $4$, the features are $(4,2,3,1)$ and the lowest-numbered neighbours are $3$, $3$, $1$ and $2$: the layer returns $(3,3,4,2)$. Equivariance would require the original output with its first and last entries swapped, $(3,1,1,2)$. The second and third entries differ: node $2$‘s output changed although nothing about node $2$ did. The layer reads the labelling, which carries no information about the graph.