Comments

You must log in or register to comment.

Smoke-away t1_irrx07t wrote

One step closer to real-time video generation.

Google Brain going crazy with the papers lately.

81

watermelontomato t1_irs0cvm wrote

My 3060 can generate an image with Stable Diffusion in around 10 seconds. If it really is 256x faster, that would be 25.6fps. I doubt the math is so clean and clear cut in reality though.

48

SituatedSynapses t1_irsltdg wrote

They will discover some unique tricks to interpolate the future frame with the previous frame's render and be able to get that over 30 FPS I bet. The biggest problem I've noticed with AI generation is the huge amounts of VRAM it needs. I really don't know how they're going to get around that and I'm very curious to see what sort of wild tricks they figure out! :)

20

dasnihil t1_irtbsir wrote

i agree, it does need more VRAM to output faster, but im more excited about upcoming videos that maintain coherency like a proper human made video, then add audio synthesis to it and we all can implement our ideas and create amazing things. even if the render takes time, still amazing improvement to have.

2

-ZeroRelevance- t1_irvlkdo wrote

Seems like StabilityAI have some ideas for how to reduce it, since they seem pretty confident about getting Stable Diffusion below 1GB of VRAM. We’ll have to wait and see though.

2

Ezekiel_W OP t1_irrrk6e wrote

>On ImageNet 64x64 and CIFAR-10, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from.

Soon diffusion models will run on smartwatches.

42

esoteric23 t1_irsymqo wrote

How long until video games are rendered this way? Simple 3D modeling for scene composition and game state, then spend your GPU budget rendering it all in an AI vision pipeline stage.

22

HeinrichTheWolf_17 t1_irt4okf wrote

The clock is ticking for game and film studios, artists aren't the only ones being replaced. One day you can think something can create it in an instant, want a full remake of DOOM 1993? Done. Want a Grand Theft Auto Game set in South Park? Done. Want a Rick and Morty based MOBA? Done.

Soon we will be able to create anything we want specifically tailored to us.

32

camdoodlebop t1_irtp4ma wrote

another interesting aspect will be NPCs with much more intelligent chat capabilities. imagine being able to have a philosophical conversation with a random character in a video game through ai. imagine a video game where all of the NPCs know they're in a game

18

Think_Olive_1000 t1_irtjb4v wrote

Will still be a market for multiplayer no matter how much ai can replicate human players i think people will still enjoy knowing the person on the other end has a real mom

9

CyberDaPlayer1337 t1_it7ywfc wrote

People already play multiplayer games blissfully unaware that the majority of the lobbies they’re in are entirely comprised of bots

1

polygon_lover t1_irvy8un wrote

Nah. AI generated assets are one thing, but AI making game design decisions is a whole other realm away.

1

Quealdlor t1_isgg8i7 wrote

I want huge open-world games with very good graphics set in some chosen anime worlds for example.

1

Ezekiel_W OP t1_irt34ut wrote

This is the reason Nvidia has put 4x the number of AI cores in their newest GPUs, they believe as I do that all games in the near future will be simulations created with AI.

11

dasnihil t1_irtceak wrote

one of the most beautiful chipsets ever built. ada lovelace.

6

-ZeroRelevance- t1_irvm1gz wrote

Realistically, we’ll probably see the first games that aren’t just tech-demos maybe late-decade, and then some games you’d actually want to play a few years after that. To be honest I think these numbers are pretty conservative, but it’s pretty hard to predict so far ahead in this rapidly evolving climate.

1

esoteric23 t1_irwemxv wrote

Yeah, seems very conservative. Seems like all we’re missing is frame rate and better frame to frame coherence and you’re set. Like, it wouldn’t be that much different than post-processing effects on emulators.

1

Zermelane t1_irs4yfp wrote

> However, a downside of classifier-free guided diffusion models is that they are computationally expensive at inference time since they require evaluating two diffusion models, a class-conditional model and an unconditional model, hundreds of times

Doesn't seem to match what I see with Stable Diffusion. One of the most popular UIs has 20 steps as the default, and that works great in my opinion.

And people have harnessed the unconditional call for an "undesired content" feature where you actually do give it a prompt, and then classifier-free guidance takes the picture away from including that prompt. That's a fairly popular feature, so losing it for faster gens would be a tradeoff, not an unqualified win.

10

DangerZoneh t1_irshywr wrote

The diffusion model is actually the easier part of an image generator. It’s the encoder model that gets you good results.

4

Saerain t1_iruyrzb wrote

This rate of software innovation is incredible.

Little did we know the last 10 years or so of GPUs have been such untapped genies sitting in our PCs.

3

visarga t1_irt4m6q wrote

An important observation to make is that it's only been demonstrated on images sized 32x32 and 64x64. A long way away from 512x512. Papers that only test on small datasets are usually avoiding a deficiency.

0

Think_Olive_1000 t1_irtjoc7 wrote

i dont see why it couldnt scale? theyre doing a like for like comparison of the speed up compared to past models' speed with 64x64 image generation

4