kross00 t1_jcnr2yo wrote
How is it different from the inpain already built in controlnet?
mikonvergence OP t1_jco5znu wrote
Which implementation do you have in mind?
When putting this together, neither the official implementation (https://github.com/lllyasviel/ControlNet) nor the diffusers pipeline (https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/controlnet) had the inpainting option built-in (and it seems they still don't?).
​
While this framework still follows the principle of injecting ControlNet features into a core SD backbone, this core backbone had to be changed to an inpainting one, to allow mask input (and since the input also includes a mask, it is not possible to just specify a different backbone source and reuse an existing pipeline). The pipeline provided in this repository StableDiffusionControlNetInpaintPipeline implements this approach and merges the interfaces of StableDiffusionControlNetPipeline and StableDiffusionInpaintPipeline, so you can provide a source image, a mask, and a control image, and also set all possible parameters to the values you like.
Viewing a single comment thread. View all comments