Well, if it makes you feel any better, we've also explored every other type of possibility that is allowed:

• scalar field models, like the Higgs field
• vector field models, like electromagnetism and the strong and weak interactions
• tensor field models, like general relativity and modifications of it
• supersymmetric matter models

These, together with ordinary matter models, are exhaustive of what the allowed possibilities are, at least for particle forms of dark matter (keeping in mind that aggregate forms, such as black holes and other MACHOs, have also been largely ruled out except for within a range of very small masses). There are also combinations of these, such as TeVeS (which gets its name from combining tensor, vector, and scalar fields).

To date, the only types of models that have succeeded at explaining all of the observational evidence are matter-based models (and possibly some of the more contrived supersymmetric matter models, I am not super familiar with those but I assume they operate similarly to matter models — what I do know is that the most minimal supersymmetric models have been observationally ruled out and to date there remains zero evidence for supersymmetry in nature ... though not for lack of trying; discovering supersymmetric particles was one of the goals of the LHC, as well as other experiments).

So, it's not so much an assumption, so much as "we've explored all of the other possibilities and this is the only possibility that actually works."

XKCD wrote a comic about this. There seems to be this popular misconception that we just assumed it was matter because that's simple. No, we "assume" it is matter because virtually every other possibility has been heavily explored and every single one of them has failed to fit all the data even in the most contrived circumstances. Dark matter is really the only game in town, it is the only model of all the thousands that have been explored which fits all of the data.