Because whatever the mass is, it isn't emitting any light or blocking any light. Everything has a temperature and normal matter should be detectable in some wavelength or another, especially in the amount needed to create the lensing effects. A cloud of dust and gas massive enough to cause the lensing would be easily visible in the radio/far-infrared spectrums.

Gravitational lensing is one way - even if we can't see the dark matter, we can notice the lensing of light passing around it if there's enough of the stuff.

The obvious example is the Bullet Cluster. It's two galaxies that collided with and passed through each other. It's been a while since the collision so there's a fair amount of space in between the two galaxies.

What's interesting about the bullet cluster is that the lensing effects appear to be focused around a point in between the two galaxies, in what appears to be empty space, rather than the galaxies themselves, implying a large invisible mass of dark matter that has been left behind since the collision.