Some shows' creators are more continually hands on - you won't have this particular complaint about any of Lindelof's shows, the Sopranos, the Wire, etc. Those are created in a similar vein to films where one director oversees pretty much everything, or enough of everything that you could call him/her the 'director' of said show if you wanted to.

It just depends on the kinds of shows you watch. If you watch more episodic stuff or dime-a-dozen 20+ eps per season types of serialized shows whose main focus is on trying to get renewed, you'll have this issue frequently. Because they do often have different people direct and oversee episodes. But if you're watching shows that're more compact and/or have a clear singular vision, you'll be less irritated by this. Miniseries especially.