My guess is that differences other than frequency response are related to harmonic distortion, and things like ringing/resonance in the headset cup, mostly.

Harmonic distortion makes it hard to tell instruments apart because pure tones already gain extra overtones which can audibly affect the character of the sound if they are above some -60 dB relative to the main tone, and multiple tones do not blend cleanly, either, but interact and create additional extra frequencies, and it is typically called intermodulation distortion. These extra frequencies could be perceived as extra noise, or timbre changes, or such, and may make it hard to tell instruments apart. It is one of the reasons why I look for harmonic distortion graphs, especially those that have separated 2nd, 3rd, 4th and so forth, as physiological measurements of human auditory system show that the masking of the harmonic distortion mostly covers the 2nd harmonic at some -40 dB level, but barely at all for the higher ones, though there is general tendency for harmonic distortion below -60 dB to be inaudible no matter where it is.

For over-ear headphones, ringing in the cup is probably visible as narrow peaks in frequency response at some specific frequencies, assuming the graph is not overly smoothed. Ringing is usually also visible as minor kinks in impedance graph as well, as the driver behaves somewhat differently at those particular frequencies, and is likely also seen as abrupt changes in the group delay and phase. So I like to see a nice flat group delay plot up to some 10 kHz, to know that there are no phase or ringing issues to be expected. Above some frequency depending on the cup's distance to headset fixture and earlobes, the measurement device itself will add all sorts of phase issues, and generally speaking the measurement above 10 kHz is not usable. For IEMs, I think the measurement reliability extends far higher, though there will be a peak for the ear canal resonance frequency where soundwave bounces between eardrum and the IEM and the exact frequency depends on insertion depth.