Most of the existing stable diffusion demos rely on a server behind to run the image generation. It means you need to host your own GPU server to support these workloads. It is hard to have the demo run purely on web browser, because stable diffusion usually has heavy computation and memory consumption. The web stable diffusion directly puts stable diffusion model in your browser, and it runs directly through client GPU on users’ laptop.

This means there is no queueing time for the server’s response, more opportunities for client server co-optimizations, and friendly for personalization and privacy.

Github page: https://github.com/mlc-ai/web-stable-diffusion

Also comes with a online demo: https://mlc.ai/web-stable-diffusion/

Comments

kalakau t1_jcn6q2v wrote on March 18, 2023 at 1:20 AM

#2,258,944

absolutely amazing, thanks for doing this!

Educational_Ice151 t1_jcn7818 wrote on March 18, 2023 at 1:23 AM

#2,258,980

Wow! 🤯 this is so useful

Shared to r/aipromptprogramming

remghoost7 t1_jcnvpqh wrote on March 18, 2023 at 5:05 AM

#2,260,417

Very interesting....

Reminds me of how some VR apps can run natively in browsers, using hardware acceleration I believe. I'm guessing this is something sort of similar to that....? Could be entirely wrong though.

Cool stuff though. Would be need to make an extension of this for A1111.... Not to diminish the work you've done, but it would probably get more exposure that way (since it's the most used Stable Diffusion front end out there).

asdf3011 t1_jcnz5d2 wrote on March 18, 2023 at 5:45 AM

#2,260,605

Going to send this to people when someone shows curiosity to trying image generation. Thanks you!

shadowknight094 t1_jco3sjk wrote on March 18, 2023 at 6:46 AM

#2,260,838

When I click generate button in Chrome on my desktop nothing happens? Is this a service worker or a network call? I don't see anything in Chrome dev tools

WASDx t1_jcogs0j wrote on March 18, 2023 at 9:55 AM

#2,261,374

How does it store the model? I assume it's not re-downloaded every time you visit the page, and I would not expect my browser to allow caching multiple gigabytes from a single domain.

race2tb t1_jcops9d wrote on March 18, 2023 at 11:51 AM

#2,261,801

I think there is a need for a p2p torrent backend for models that needs to exist now.

crowwork OP t1_jcourze wrote on March 18, 2023 at 12:43 PM

#2,262,088

Replying to WASDx (#2,261,374)

models are stored in the browser cache via an explicit browser cache api. So the model is cached once.

WASDx t1_jcpeej0 wrote on March 18, 2023 at 3:18 PM

#2,263,211

Replying to crowwork (#2,262,088)

Explicit as in requiring user approval? My web development skills are really out of date, can you link to the API?

crowwork OP t1_jcpojd6 wrote on March 18, 2023 at 4:27 PM

#2,263,758

Replying to WASDx (#2,263,211)

Explicitly means we need to write js code to request the cache as oppose to browser's automatic caching. https://developer.mozilla.org/en-US/docs/Web/API/Cache

LetMeGuessYourAlts t1_jcq33nb wrote on March 18, 2023 at 6:05 PM

#2,264,447

Replying to remghoost7 (#2,260,417)

The Unreal Engine let's you compile to browser with basically a few clicks so that might be how they're doing it and abstracts away a lot of the hard parts.

remghoost7 t1_jcq8emm wrote on March 18, 2023 at 6:41 PM

#2,264,676

Replying to LetMeGuessYourAlts (#2,264,447)

Ah, it was made with unreal.....? I didn't see that.

I always love adaptations of video game engines. One of the reasons I've been a huge fan of Unity for years. It's essentially just a wrapper for C# code with a pretty interface.

marcus_hk t1_jcrdufd wrote on March 18, 2023 at 11:46 PM

#2,266,490

Replying to race2tb (#2,261,801)

For weights, yes, and for inference. If you can decompose and distribute a model across enough nodes, then you can get meaningful compute out of CPUs too — for instance for tokenization and smaller models.

marcus_hk t1_jcrgwqm wrote on March 19, 2023 at 12:09 AM

#2,266,614

Just browsing on my phone and haven’t dug deep yet, but in the notebook it says that build.py targets M2 by default but can also target CUDA. What about CPU?

I’d love to see a super minimal example, like running a small nn.Linear layer, for pedagogical purposes and to abstract away the complexity of a larger model like Stable Diffusion.