In fact, this is only part of the story. Degeneracy pressure contributes pretty meaningfully to the "solidness" of solids, so insofar as the idea of "empty space" even makes sense, atoms are not "mostly empty".