Yes.

Text to Video is already happening, and Text to 3D Environment doesn't seem particularly more difficult.

In many ways it may be easier, since 3D elements can be broken down into components and "understood" by an AI.

For example, a 3D room has a floor, walls which connect, a ceiling. Now generate varieties.

Or a 3D character: A body has known elements. A face has specific parts. Look at Unreals Meta Humans, which are ultra-realistic faces with feature sliders.

Much of this already exists within the game building and level building world. It's just been under-trained by modern AI practices.

NVIDIA is working on it, as are many others. I wouldn't be surprised if we see mind-blowing examples of this working in months not years.

All the old school text based games could be revived with AI graphics creating visuals for what's happening

Yeah, the "input-to-output" progression I see coming out over time is:
* Text-to-text
* Text-to-image/audio
* Text-to-3D model
* Text-to-animation/video
* Text-to-interface/game
* Text-to-physical object (through 3D printing and other means)

There will be some overlap though, so this is not necessarily a strict ordering. And the "text" input could also be replaced with voice or some other modality. Image-to-game, 3Dmodel-to-physicalObject, etc.

Probably won't be text, you'll just have a conversation with an agi, using ar/vr to direct aspects of the games creation in 3d space

Im sure we will see the equivalent of 1980s Atari AI generated games soon enough.

Just to add a pessimistic take on that : I think we're still very far from that, this really sounds like something that could only be made by an AGI (for example anything that includes human interactions is basically mini turing tests if you want it to be realistic).

But I don't see a reason for it not to happen eventually, maybe in 20 years or so if we're lucky ?

Imagine Star Trek. 🤔

Well, at that point it really blurs the distinction between a 'game' (characterized by some constrained set of rules and goals) and more of an 'AI-generated experience'. Not that there's anything wrong with the latter, it's just a bit of a terminology issue.

And yeah, I expect that before the end of this century we will be able to jack into our own personal Matrix and experience pretty much any environment and narrative imaginable within the realm of human cognition.

Yup, wouldn't surprise me if someone is already training a large language model on unreal engine so you can just ask it to do something and it creates it for you in the engine. This is definitely a near term reality and by near term I'm thinking less than 2 years for individuals to create indie quality games.

Yes

Depends if you mean a fully fleshed AAA game or pong. We can probably already do pong. But the leap from video to AAA game is much greater than the leap from image to video. It will essentially mean the AI is capable of writing 100K+ lines of coherent code. So basically I don't think we'll get it until we get AGI.

I have a ton of non-fiction stories, that I'd absolutely love to bring to life, at moment's notice.

Sounds like a much better work-flow, than current game-dev methods!

I asked this over at /r/gamedev and people was furious because "only people are supposed to make games" or "I enjoy the process of making a game so we don't want AI" and similar sentences. Completely disregarding the advantages AI could bring like faster production rates and cheaper games but yeah no, that isn't as important as them being the one making the product.

Oh customers don't want to wait 8 years for the next game? well too bad.

I know this is not really what you're talking about, but Character AI has some really fun ai-powered role playing games. You also have the ability to create your own games or characters if you want!

Text to Game will be a huge, if a solo dev can create a AAA tier game that is actually fun to play, from scratch? That's when shit gets real.

Although I think AI completely generating a game from scratch is likely to produce a lot of mess and incoherence if it's not using AGI. In the nearer term, I would expect game developers to act more as directors, in the sense that they will assign particular tasks and modules to an AI, which the director will then stitch together into something coherent and compelling.

For instance, creating dialogue for NPCS might be a task of manually designing a dialogue tree, then using AI to generate the text, curating and pruning portions of it until it looks right, after which you'd generate your voices, facial animations and design of the NPC itself all via AI.

Exited for this lucid dream/drug trip level of simulation

For a more level headed take :

Think about how complicated a game is. Enormously huge environments, behavior of npcs and objects fitting well defined rules, multiplayer synchronization.

So even with pretty advanced AI, getting A game that can be played is a very different ask from "give me a game like gta 5 but better"

Moreover, even if a machine could in fact give you a "better" knockoff of gta 5, without many people playing the game and giving the AI feedback on where to improve it still might not be a very good game.

I think it will happen within 10 years - first Indie developers will make AAA games and soon after everyone can do it.

More than what we already can do with codex?

Is here now just isn’t accessible

I'm more interested in text to will to live

“computer, i want to experience a plotline set in the world of the My Hero Academia anime where im a Nomu and im rebelling against my villainous masters and the heros help me become a law-abiding civilian. and i start living in a government funded grouphome for rehabilitated monsters. and i fall in love with a fellow monster.”

ERROR: COPYRIGHT INFRINGEMENT

“okay, um. do what i just said, but replace all the copyright-infringing elements with original ripoff material; as similar as u can get away with”

COMMAND ACCEPTED. CREATION TIME: 40 MINUTES.

“ok cool im gonna play my ps5 in the meantime”

Right, text to "game".

Remember the Trek episode where we get to see Barkley's custom holodeck scenario with Deanna Troi's expy playing the role of the goddess of pleasure?

That kind of game?

I see it happening and I also see intellectual property holders having a meltdown. We are seeing that now with AI art. Quite a few popular modern artists are seeing AI art that they didn’t create and don’t own but that is based on their own unique style. What happens when you can tell an AI I want to use a CoD style game engine but set in middle earth and using stormtroopers riding pterodactyls as air support. The entire world of pop culture and human history can be appropriated and repurposed for game play. It is going to challenge the concepts of ownership.

I think so, yeah. This whole field seems to be doing incredible things. I don't think it would be more than a couple years to have, at worst, a rudimentary model that can do this.

IMO, it will most likely require AGI. It is almost the same task as a fully automated software engineer.

In the mean time we have my game called AI Roguelite on steam to entertain us until then

Edit: some might wonder why it would take so long if we already have auto video etc. This only becomes clear when you start thinking about how to build this kind of game (i.e. what would AI Roguelite 2 look like and how would it be built). It is not enough to auto generate video or models or even animation. With those 3 we could get something like No man's sky that's much more visually detailed but not necessarily better gameplay. The game needs more info like what each attack/ability actually does, creative enemy abilities/behavior, what the effect of an item should be (which are often very open-ended and shouldn't simply be +dmg) etc. When all these are taken into consideration it basically requires AGI

No. Text to image is very different in complexity than creating a game which has a story telling, different levels, game mechanics, level balance.

Can AI help in creating a game in some parts like texture design? Yes.

[deleted]

I hope one day it can be like aidungeon but it generates everything in VR as you interact with the game world. That would be the dream for me.

If players can give the AI fedback about what they're experiencing in real time, it can learn what works and what doesn't for building 3D worlds/games for us to experience at a faster rate. E.G. We're hooked up to a feedback device that lets it know if we like what we're seeing. Or just a few buttons on the screen we can press to let it know what we think. Click on the object in the room that's out of place or looks weird and let it know that's not right.

Something I think about sometimes is "the infinite library", where you have infinite books, each book simply containing some sequence of characters.

Every conceivable book is in the infinite library, but finding anything is basically impossible. Somewhere in there is a book with your life story, all the works of Shakespeare, and every math solution, and extraordinary books which no one ever wrote. But there's no known catalog.

Diffusion models feel like a way to filter out most of the garbage(and likely a lot of good items too) and pointing to stuff that isn't pure nonsense.

Makes me wonder if we could train on NES roms or stuff like that.

you have to be so descriptive with it, I can imagine people thinking "screw this, I would rather just code instead" hahaha

Like AI Dungeon 2D?

Instant thought manifestation

It will be awesome in opening development to so many more people, but people are sleeping on upscaling old games and how awesome it will be. It's realistic that this can run in real-time and so will be highly customizable.

Just imagine playing games like Chrono Trigger or the first Metroid but they look as good or better than the best anime out there or Into the Spiderverse. So many well-crafted experiences that aren't worth playing for most right now, they can be made to look photorealistic or with personal artistic styles, but just generally look beautiful and crisp like modern graphics.

Same with old movies, even sports. Lots of sports classics locked away because it looks like it runs on a bad potato. So much potential to look for the best footage of sports legends and have UHD of the best of their career. Too laborious by hand, but for a machine it's trivial once the various parts can be hooked together. As long as it's digitized, it can be rebuilt. We have the technology. Well, soon.

It can even go wilder, like old movies reimagined. Just a bit or a lot. Like what would Gone with the wind look like if Tarantino directed it? Or lightly update the graphics and styles to something more modern in terms of cinematic style and camera work. Or replace an actor. That one will be a teensy bit controversial.

Lots of potential for people to do remakes of series or movies. Like remove an entire plotline from this series, and this character dies in season 2 instead of continuing. It's the same series, just that one mistake is entirely gone. Or reworked, the timeline presented in a way that makes more sense. Or remove all the filler episodes. Instant remix from a simple instruction. Unlimited potential.

Yes. I'm a game designer and in part it's already happening.

I'm just going to leave this here.

By 2026 you’ll be able to generate a simulation of any environment, real or fiction, and populate it with NPCs of any kind. You want human NPCs? Sure, they’ll all be instantiated with a next-gen language model and can carry a unique conversation indefinitely. You want never before seen alien species? Why not, go ahead and breed & evolve them while you’re at it.

Yeah I'd like txt to Game with Unreal Engine lol! I'd make:

Megaman Starforce 4 Tales of Rebirth, Symphonia, Abyss, and Innocence remakes Infamous second son 4 aka infamous second son 2 And an Assassins Creed set in the American Civil War!

Right now you can create a game from text using your natural intelligence. And within a few days. Look at Inform 7.

r/interactivefiction

With enough time, resources and will, everything will be possible.

I wouldn't be surprised if the next iteration of text generators (in a few months) have the ability to do fully fledged adventure text games by writing the story first, with all it's possible outcomes and links between the text segments, then putting the segments inside a traditional text adventure shell program. Even if that AI then pipes that story or story portions into another AI text adventure software generator to output the program. If passing the Turing test is a few months away as hinted, then these would be possible and refined by the end of next year.

As to more advanced games? We'll be seeing feedback loops and exponential returns over the next few years that will see the development of AI generated games parallel the development of the history of video games, but in accelerated time; whether this will hit open world VR levels before we hit AGI (and everything accelerates) will determine if it is an exponential curve or a hyperbolic one.

In another thread people are speculating about proto agi in 2023.

I'm not sure if they realize that a proto agi should be able to do what you are asking, among a lot other things.

A proto agi should be able to create every kind of software and iterate over it according to your instructions.

When will this be possible? Hard to say but likely in the next 20 years.

I would love to play The Passion of the Christ video game as the Mario Bros.

Saw that somewhere HhHhHzhHmMmmmmmatrix xd

It will not, the companies creating these technologies would never allow you the opportunity to create the products you imagine. they'll use aggregated data to sell you the products you've imagined.