> However, a downside of classifier-free guided diffusion models is that they are computationally expensive at inference time since they require evaluating two diffusion models, a class-conditional model and an unconditional model, hundreds of times

Doesn't seem to match what I see with Stable Diffusion. One of the most popular UIs has 20 steps as the default, and that works great in my opinion.

And people have harnessed the unconditional call for an "undesired content" feature where you actually do give it a prompt, and then classifier-free guidance takes the picture away from including that prompt. That's a fairly popular feature, so losing it for faster gens would be a tradeoff, not an unqualified win.