> CNNs provide the inductive bias to prefer functions that handle translation equivariance

There's some interesting bodies of work to inductive biases in CNNs, such as "Making Convolutional Networks Shift-Invariant Again". Really interesting stuff!