That's a super interesting point, and I think it's an important one.

As an idiot on the interweebs, I think the opposing argument might be something like this:

Researchers, like everyone else, have a limited amount of time and energy. If they focus their resources heavily on the specialized knowledge and advancing the field, they can push the limits farther, faster, increasing the sphere of knowledge; others can follow behind and find ways to share the knowledge and make it more accessible.