I'd imagine the more steps in between "generate graphics" and "display" add a considerable amount of latency. From my understanding we're already at the point where having the CPU physically close to related chips (memory's one, IIRC) makes a difference. Could be wrong, but from my understanding the last thing you want to do is throw a bunch of intermediate hardware/steps in the process if you can avoid it.