Yes, the ideal headphone would be a minimum phase system. But manufactures go to a lot of trouble to design them to be as close as possible to minimum phase, and still come up short. For one, minimum phase systems don't have nonlinear distortion. Headphones do.

The "headphones are minimum phase systems" arguments tend to remind me of the old joke about the theoretical physicist hired to consult on how to improve milk production efficiency on a dairy farm. When time to report his solution to the farmer, he begins "First, we assume a spherical cow...."