Viewing a single comment thread. View all comments

Haycart t1_jdu7hlp wrote on March 27, 2023 at 5:55 AM

Reply to comment by visarga in [D] GPT4 and coding problems by enryu42

Oh, you are probably correct. So it'd be O(N^2) overall for autoregressive decoding. Which still exceeds the O(n log n) that the linked post says is required for multiplication, though.