Since CNN weights are often 4D tensors, WxHx(#inputs)×(#outputs), they're hard to visualize directly. Instead, the trick is to ask what input images to the model fully activate the queried node and none of the other nodes in the same layer. There's a Keras example script that does this. The generic term is "deep dream."

Wouldn’t a deconvolution operation (backprop basically but without updating gradients) on any layer after the network has been trained show u what features activate that layer?