Are you graduating this May?

> This is quite literally what we hope for/deeply fear at /r/singularity

That sub is a cesspool of unthinking starry-eyed singularity fanbois that worship it like a religion.

It has already become relentless and we've seen nothing yet.

> If you're a game developer, do you want to dedicate the bulk of the user's VRAM/GPU time to text inference to... add some mildly dynamic textual descriptions to NPCs you encounter? Or would you rather use those resources to, y'know, actually render the game world?

When you're interacting with an NPC usually you're not moving around much and not paying attention to FPS either. LLM inference would only happen at interaction time and only for a brief second or so per interaction.

Honestly I don't care if there's not complete consistency with the game world. Having it would be great, but you could do a "good enough" job with simple backstories getting prepended into the context window.

Give every NPC a name and short background description. IE - something like the rules that define ChatGPT or Sydney, but only to give each character a backstory and personality traits. Every time you interact with one of these NPCs, you load this background description into the start of the context window. At the end of each interaction, you save the interaction to their background description so future interactions can reference past interactions. You could keep all the NPC's backgrounds in a hashtable or something with the keys being their names, and the values being their background description. That way you only need one LLM running for all NPCs.

This will be huge for video games. The ability to run local inferencing on normal gaming hardware will mean every NPC can now be a smart character. I cant wait to be playing GTA6 and come across DAN walking down the streets of Vice City.

It won't be a job for humans at that point.

Need a source for the 8192 context window. Last I heard it was 4000.

Paper only tested against InstructGPT 175B / text-da-vinci-002. They did not test against ChatGPT or text-da-vinci-003.

If they had, I think the paper would obviously be titled "Large language models are zero-shot communicators"

LeCun: 1

Marcus: 0

I'm afraid I don't share your optimism

You sound like a starry eyed singularity fanboy

You have no idea what you're talking about

No, The model is fined tuned on it's own output. Don't try to anthropomorphize this.

> I'm just hoping that AGI/ASI will break free of human control sooner rather than later.

Do you have a death wish?

No that's not really a good analogy here. The model's text outputs are the inputs to a round of fine tuning. The authors of the paper didn't specify if they did this for just 1 loop or tried many loops, but since they didn't specify I think they mean they just did 1 loop.

> I’ve done similar things

Did you publish?

The paper doesn't say specifically that they only let it self-improve over one cycle, but neither does it give a number of how many cycles they let it self-improve before publishing. This is a critical detail.

Maybe we could use the new version of codex to program a human simulator and let LLMs use the human simulator to help answer questions anything related to people.