There's a reason the series started with that scene (it did in the book too). That was supposed to indicate that the White Walkers were ultimately the important threat and the war of the five kings was, quite literally, a game of thrones in comparison.