GPT characters in games

Submitted by YearZero t3_126rcq0 in singularity

This is what I really want. Generating worlds from prompts is cool. But I’d love to see more intelligence in game characters. Maybe within certain parameters of the story and personality and role of the character, but with freedom to improvise. Imagine playing a Skyrim type game or Cyberpunk 2077, but have a different experience each time? Maybe you can make certain characters into friends or foes based on how your conversation goes.

Maybe the characters can have decision making based on the same principles. A character can follow you, attack you (and decide exactly how, with what, and to what degree), or convince its friends to join it for the attack etc.

I think this should be the next real frontier in gaming. Graphics are already pretty damn good. Physics certainly could use some work but it’s coming along. Destructible environments and such are hard but also not always needed for every game.

But AI has been stuck in decision trees for decades. We aren’t making clear progress in game character AI like other stuff, and we need a proper leap. Games would become way more engaging.

The question is - can modern CPU and GPU handle all the inference, especially with multiple characters all interacting at the same time? While also handling the rest of the game physics and graphics. I’d sacrifice the latter for an experimental attempt at the former.

73

Comments

You must log in or register to comment.

WonderFactory t1_jeaotp1 wrote

I'm actually adding ChatGPT NPCs to the Unreal Engine 5 game I'm developing at the moment, it's a rogue like set in a post singularity world, gameplay is similar to Hades so there's plenty of dialogue in the game. at the minute it's difficult to get a model to run on the local PC so I'm using OpenAI's API. There are challenges like latency while you're waiting for the API to return, it's also quite expensive so releasing a free demo of the game is out of the question. It could potentially cost several dollars per user in API fees over the life of the game so that will of course limit your pricing flexibility, you can only reduce the price so much in steam sales etc. I'm hoping though that inference costs will come down by the time the game is finished.

I haven't posted any footage with the GPT dialogue added to the game but I might post it here in a couple of weeks.

28

sumane12 t1_jeb1wh1 wrote

>I'm hoping though that inference costs will come down by the time the game is finished.

Genius. This is exactly what will happen, and I'm glad someone has the forethought to develop the product, before the underlying technology is ready, because it will be there.

13

psdwizzard t1_jebn0zq wrote

you may be able to run a custom version of Llama instead of using GPT. You could even train in on GPT3.5. Just tell GPT its a character from the game, build your data set from its answers and use it like Alpaca does.

5

maven_666 t1_jed51is wrote

Licensing issues with llama seem to make this not workable but I expect a fully open version soon.

4

alexiuss t1_jeb63mr wrote

Why not release it so user can enter their own API to make it work? I'm super interested in helping you develop this stuff, pm me.

2

2Punx2Furious t1_jecz2b2 wrote

I think you could get around the latency issue by having the generated dialogue come in form of letters that you receive in-game, which would feel a lot more natural than a slow conversation. Or have some cutscenes in between the prompt and the answers. As for the price, it should probably be an optional setting, and maybe the price should be offset by a subscription or ads, as much as I hate them, but in this case it would be difficult to do otherwise, unless you plan to foot the bill of your users forever.

1

FoniksMunkee t1_jed4826 wrote

How do you restrict the AI's answers to say further a plot point or stay in character?

1

MassiveWasabi t1_jeb55ha wrote

Check out this paper that Microsoft researchers just released. Among a ton of other cool things, they talk about how this new model they are working on, TaskMatrix.Ai, will be able to take control of "AI teammates" in team-based games, and that you can give each individual teammate different tasks in order to carry out a complex strategy. This seems like the next step to having truly dynamic characters controlled by AI, hopefully so dynamic that they seem completely real.

10

Mortal-Region t1_jeaia9q wrote

>We aren’t making clear progress in game character AI like other stuff, and we need a proper leap.

Problem is, it's such a huge leap going from decision trees to autonomous agents who can form their own objectives and plans with respect to other characters and the environment. It's pretty much the same problem that brains evolved for. LLMs aren't agents, so I think they'll end up as automated dialogue generators, with the overall storyline still being "on rails".

6

ReignOfKaos t1_jebvoyd wrote

In theory, reinforcement learning could solve this, but specifying the reward function is very hard and if you need human evaluation then it is very slow. I think it’s not quite as theoretical of a solution as evolutionary algorithms, as it’s been successfully applied to create very capable game playing agents before, but I think it’s very difficult to beat behavior trees or simple utility systems when it comes to creating characters that are fun to play with.

2

[deleted] t1_jecp7fs wrote

[deleted]

2

FoniksMunkee t1_jed4er2 wrote

So it's a hybrid approach?

2

[deleted] t1_jed7r25 wrote

[deleted]

2

FoniksMunkee t1_jedcqe9 wrote

It's interesting - I work in AAA games and work with a lot of clients. They are deathly silent on any kind of AI integration so far. So it will be interesting to see when this starts coming down the pipe.

3

[deleted] t1_jedeic5 wrote

[deleted]

1

FoniksMunkee t1_jediwl9 wrote

Yes, that's a good point I didn't think of is API costs.

The other issue is games kinda move slowly in some respects. There are games that started before ChatGPT was commonly known that wont be finished until after ChatGPT 5 is out. And there is no way these tools will be integrated during that process.

They would also have to convince Sony, MS and Nintendo to have their SDK's in a model. I don't think MS will necessarily have a problem with that... but there's a ton of third party libraries that would need to come on board, not to mention how you deal with existing legacy code. There are still more companies in the AAA industry with custom engines than are using commercial engines like UE4/UE5.

Then comes the RND, then comes the new game... so what, 5 years before we see wide spread adoption in the AAA market?

2

[deleted] t1_jedlka4 wrote

[deleted]

1

FoniksMunkee t1_jedm0mj wrote

There will be teams that are starting games today - that might look at ways of using it. I Know of some teams using it for art pipelines (but mostly inspiration / mood boards). And there will be some tools like Photoshop and Blender etc that will mean we will see it appear in the pipelines earlier.

But as far as wide spread penetration - it will probably need to wait for teams to start new projects. So obviously, a team starting today may choose to do something already. But widespread use? It's way too risky. It's hard enough to justify an upgrade from UE4.27 to UE5.1. I can't imagine up-ending an entire gameplay system just to integrate AI would be an easy sell.

And while I won't be the first to know when it starts happening... I will be early on because of where I sit in the pipeline. All I hear right now is crickets.

1

azriel777 t1_jed9mfh wrote

Some combo system. The NPC's are like skyrim, on rails with some randomness so it gives the illusion of being alive, but when the user interacts with them, the AI takes over and fleshes them out to give them life and when you leave them they revert back to what they were, perhaps with some modifications to their rules and behavior depending on what the user did. The AI could also randomly take over NPC's to generate interesting things for the user to see or generate quests for the user. The potential is there, just going to take some trial and error to figure out what works and doesn't.

2

megadonkeyx t1_jeb4ecc wrote

It's almost inevitable that it's going to happen, in a few months you won't be able to take a leak without an AI telling you to wash your hands. (Exaggeration)

Anyhow, technically doing this on the local machine is the way to go, checkout the 7b llama models that can be quantized down to 4bit. Something like that trained for games, run it on the background on a single thread.

It would not be super fast but it could chug away in the background analysing the game state and making decisions.

4

FoniksMunkee t1_jed4g87 wrote

Oh shit... I just realised the future of Japanese toilets.

2

No_Ninja3309_NoNoYes t1_jebobu3 wrote

If we all sign the LAION petition, we will get whatever we want. But someone will disrupt the game industry soon for sure. I hope they get rid of the text prompts too. Why can't we just upload some rough sketches instead? Or other files.

1

genericrich t1_jeahu1l wrote

I don't think it will be useful for games, since games are storytelling medium and introducing randomness that can't be well-controlled into stories makes them into bad stories.

I don't think it will be feasible to work well enough.

−9

3z3ki3l t1_jeakjny wrote

But it could be used for background characters. That way it’s not the same line every time you run into a character.

So every guard in Skyrim didn’t take an arrow to the knee. Could be as strict as generating some other injury, or loose enough to provide an entirely different excuse for not joining the war.

15

genericrich t1_jeakxai wrote

Sure, if you can control it. The problem is that the developers need to be very sure that they aren't introducing a bug or meandering, meaningless, dead-end side quest. That would be hard to verify, IMO, with an AI-generated content layer.

I guess we'll find out sooner or later, because I am sure these will be rolling out soon.

0

3z3ki3l t1_jeanw6e wrote

Eh. Skyrim already has generated side-quests. It creates characters for you to go kill, and items for you to steal. I don’t think a little bit of dialogue, whether it fits perfectly or not, would break that much.

3

genericrich t1_jeaoj5c wrote

OK, if you can verify that it won't become racist or introduce another problem that a AAA game studio won't want in their game, then go for it.

0

3z3ki3l t1_jeapp2j wrote

Well, they solved that with ChatGPT already, so no issue there. Especially without player dialogue input.

6

koa_lala t1_jebfhor wrote

You clearly have no fucking idea what you're talking about.

2

genericrich t1_jebntv6 wrote

Uh, ok. (Old man grumbles about having worked at two AAA studios and several small game companies, including one he co-founded. Started in QA and ended as lead game designer before leaving the industry for more lucrative work.)

1

WonderFactory t1_jeap8lf wrote

I think it can work, I'm trying to get it working in a game I'm developing at the moment. You have to have a mix of randomness and structured story telling. I literally have to say to ChatGPT, the user is saying xyz, reply to the user but try to work this plot point into your reply.

3