>On ImageNet 64x64 and CIFAR-10, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from.

Soon diffusion models will run on smartwatches.

One step closer to real-time video generation.

Google Brain going crazy with the papers lately.

My 3060 can generate an image with Stable Diffusion in around 10 seconds. If it really is 256x faster, that would be 25.6fps. I doubt the math is so clean and clear cut in reality though.

> However, a downside of classifier-free guided diffusion models is that they are computationally expensive at inference time since they require evaluating two diffusion models, a class-conditional model and an unconditional model, hundreds of times

Doesn't seem to match what I see with Stable Diffusion. One of the most popular UIs has 20 steps as the default, and that works great in my opinion.

And people have harnessed the unconditional call for an "undesired content" feature where you actually do give it a prompt, and then classifier-free guidance takes the picture away from including that prompt. That's a fairly popular feature, so losing it for faster gens would be a tradeoff, not an unqualified win.

[deleted]

The diffusion model is actually the easier part of an image generator. It’s the encoder model that gets you good results.

They will discover some unique tricks to interpolate the future frame with the previous frame's render and be able to get that over 30 FPS I bet. The biggest problem I've noticed with AI generation is the huge amounts of VRAM it needs. I really don't know how they're going to get around that and I'm very curious to see what sort of wild tricks they figure out! :)

How long until video games are rendered this way? Simple 3D modeling for scene composition and game state, then spend your GPU budget rendering it all in an AI vision pipeline stage.

This is the reason Nvidia has put 4x the number of AI cores in their newest GPUs, they believe as I do that all games in the near future will be simulations created with AI.

An important observation to make is that it's only been demonstrated on images sized 32x32 and 64x64. A long way away from 512x512. Papers that only test on small datasets are usually avoiding a deficiency.

The clock is ticking for game and film studios, artists aren't the only ones being replaced. One day you can think something can create it in an instant, want a full remake of DOOM 1993? Done. Want a Grand Theft Auto Game set in South Park? Done. Want a Rick and Morty based MOBA? Done.

Soon we will be able to create anything we want specifically tailored to us.

i agree, it does need more VRAM to output faster, but im more excited about upcoming videos that maintain coherency like a proper human made video, then add audio synthesis to it and we all can implement our ideas and create amazing things. even if the render takes time, still amazing improvement to have.

one of the most beautiful chipsets ever built. ada lovelace.

Will still be a market for multiplayer no matter how much ai can replicate human players i think people will still enjoy knowing the person on the other end has a real mom

i dont see why it couldnt scale? theyre doing a like for like comparison of the speed up compared to past models' speed with 64x64 image generation

another interesting aspect will be NPCs with much more intelligent chat capabilities. imagine being able to have a philosophical conversation with a random character in a video game through ai. imagine a video game where all of the NPCs know they're in a game

This rate of software innovation is incredible.

Little did we know the last 10 years or so of GPUs have been such untapped genies sitting in our PCs.

Seems like StabilityAI have some ideas for how to reduce it, since they seem pretty confident about getting Stable Diffusion below 1GB of VRAM. We’ll have to wait and see though.

Realistically, we’ll probably see the first games that aren’t just tech-demos maybe late-decade, and then some games you’d actually want to play a few years after that. To be honest I think these numbers are pretty conservative, but it’s pretty hard to predict so far ahead in this rapidly evolving climate.

Nah. AI generated assets are one thing, but AI making game design decisions is a whole other realm away.

Yeah, seems very conservative. Seems like all we’re missing is frame rate and better frame to frame coherence and you’re set. Like, it wouldn’t be that much different than post-processing effects on emulators.

[deleted]

Wow, source?

Emad (the guy in charge of StabilityAI) has been saying on twitter that he thinks they can get Stable Diffusion under a gigabyte of VRAM for a while now. Here’s one of those tweets.

Thanks!

I want huge open-world games with very good graphics set in some chosen anime worlds for example.

People already play multiplayer games blissfully unaware that the majority of the lobbies they’re in are entirely comprised of bots

I doubt they want their actual friends replaced is my point.