The greatest analogy I heard in that regard is to see the CPU as a couple of professors, each calculating difficult equations, while the GPU is a large group of toddlers each coloring a square.

Of course the professors could also color the squares, but they would just waste their potential since they are able to perform much more difficult tasks. It's simply more efficient to let the couple of professors do the hard work, while the large group of toddlers works on coloring the squares.