Mastering Inpainting with Stable Diffusion: Fix Mistakes and Enhance Your Images

in today's video I want to talk about in paint and how you can fix mistakes and improve your images with stable

diffusion I am using the stable diffusion Forge interface for the model checkpoint I prefer the Juggernaut XL

Version 9 with the sampling method DPM plus plus 2m caras 30 sampling steps a size of 1024 pixels and a CFG scale of

seven I will start with a cinematic photo of a gisha in a futuristic interior

and after I hit the generate button I get something like this I usually keep generating until I

get something I like with fewer mistakes uh I found a seed that kind of has one hand good and one hand bad so I can

demonstrate in painting on it under the image you have a button that lets you send the image to the imageo image tab

here I changed the denoise strength to around uh 0.6 or 0.65 and then I try to generate

um a version of it now the generation looks the same because when I used a custom seed it took that seed also to um

the image to image tab so I will click on the dice icon to get a random seed instead and I will generate again now

with the imageo image feature it changes the entire image sometimes it changes the face sometimes the dagger but maybe

I like those and you know I only want to change a portion of it for that we have the the inpaint option under the image

you can see it says copy to and a button named inpaint click on it to copy that image to the inpaint tab uh in the top

left corner you have a little uh I button hover your cursor over it to see what shortcuts you have for the inpaint

tab I like to use S to go to full screen and then with alt and the mouse wheel I zoom in after that I paint the area I

want to change like in this case that hand now the way I want it is a little more difficult to get right but uh with

enough tries it can be done in the prompt I include the style so in my case a cinematic photo then I describe

exactly what I want to see in there such as a hand pointing to the camera with a finger I wanted it to look like the

gisha is pointing to the viewer as if she's saying you are next here we have a lot of options but most of them work at

default mask blur refers to how blurred the edge of the selection you painted is uh for mask mode you have two options

inpaint mask will change the area of the interior of the selection and in paint not masked will keep the selection like

in our case the hand and paint everything else most of the time I use inpaint mask

I find not masked useful if I make a selection of a person's face and then when it generates it keeps that same

face and changes everything outse I use fill when I want to remove something from the image because it

fills it with the color of the image and you can mention in the prompt what you want there um like a wall a color or

something if I try to use it on the hand you can see it's not very useful because it kind of flattens everything so for

this case the best option is to mask the original because we have some shapes there and we want to make it look better

like a hand by default it was set to inpaint the whole area which means you want to

describe the whole scene however i' prefer to in paint only masked that way the resolution of the image is

concentrated on that spot so I can get a better quality image for example those 10 to 2 24 pixels are dedicated only for

the hand not for the entire image now we have a lot more details there so we got the right settings we just need to

generate a few times to get different seeds until we find one that works for us uh when you made the selection it set

a bounding box around that selection however we can expand that selection to look at other places in the image so it

can understand the the context better and create better proportion and scale to fit the scene better I just add tiny

small dots that will expand that bounding box if I generate again you can see that it now looks at this portion of

the image however since there are tiny dots there it will not change much and most of the changes will be on the

actual hand now when you get a better version you can drag that to inpaint so it looks at the image and sees the

shapes better I will undo those spots to get the bounding box back on the hand since we have a better version when it

comes to in paint like any generation it's not perfect so you have to keep playing around with settings until you

get something you like after a few more tries I got one hand that could work I will drag that new image to the in paint

click on the Eraser icon to remove the selection I will make the brush bigger and paint the gaseous face when I make

the selection I try to include a little bit from the hair so it can see better what is around and try to blend it

better it doesn't have to be perfect here I tried to click s again to go out of full screen but it didn't work

because I was on the brush so I had to click outside then on S to return to normal since we have a different

selection don't forget to change the prompt also so I will make the prompt cinematic photo of a gisha face as you

can see we got a different face now but we can do more we can make it smile if we want or you can also make it cry if

that fits better with your vision I will try one for a serious expression but I think I like more The

Confident mood let's try another example of a bunny in the desert and see what changes

we can make within paint send it to the image to image tab then copy the image to inpaint press s to go in full screen

change the brush size and paint the bunny head on the head I go over the edge so it blends with the rest but for

the base where the neck comes I try to give it a shape on how I want my generation to look like in that area for

the prompt I describe what I want to see so I will add a robotic bunny head with a Sci-Fi look for the settings I'll

choose inpaint masked with masked content is original and inpaint area is only masked I'll set the doo strength to

0.6 and hit generate as you can see now we have a bunny with a robot head I don't like how it blends with the body

so in this case you can generate again or play with the denoy strength or let's blur the edge of the Mask a little and

see if we can get a better version it looks like it Blends better now we saw how we can modify different subjects but

what if we want to remove something I have here a toy boat on a pool I will send it to image to image and then in

paint and I will select the entire boat with a brush also in this case I have to select the shadow if I want to get clean

water without the boat I remove the boat from The Prompt then use the same settings as usual with Mast content

original and in paint only Mast Deno set to0 .6 I hit generate and got a different version of the boat instead of

only water why did this happen because with original masked content we keep whatever was there originally and get a

different version to remove it you want to use the fill option that will fill with a similar color of the

image so now when we generate again we get some nice water we just have to play with mask and denoy settings or get a

few tries to get it to blend better we changed and removed now it's time to add something um I will use this

empty desert for this example I will paint a selection about the size of my subject I want to add then I'll use the

same settings with original and 0.6 denoy strength We'll add a cute bunny Cowboy In The Prompt and hit generate

sometimes you get nothing and sometimes you get uh just a tiny bit of that subject there that's because it didn't

have shapes and forms to work on to add something there we choose the third option called latent noise if I hit

generate look what happens we get some noise it's like the most beautiful alien abstract okay okay it's not what we want

for the result but it's the right settings we just need to adjust something first for this option you want

to use a Deno strength of around 0.95 to work well uh we finally got our Cowboy bunny

there but because our selection was so big big the mass blur is small compared to the selection so I will increase the

mass blur to see if we can get it to blend better now it Blends better but the

selection is too small for the bunny to fit uh however that's not a problem now that we have something there like the

overall shape of the bunny we don't need the latent noise anymore so what we can do is drag this new image to in paint

and make a larger selection so it goes outside of the bunny then we reduce the denoise back to 0.6 and change the

masked content to original now when we generate again we should get a better version of the bunny I will hit generate

once more to get a better version of the gun one more example for you let's see what I did in the prompt um the best way

to get good hands is to not have them visible so put them in the pockets or behind and we'll have fewer mistakes to

fix first let's fix the face of the woman I am making a big selection and in this case I will keep the same prompt

and the same settings as I usually do I will hit generate and I will get a better quality face now I will drag this

new image back to inpaint and move to the next thing what if I want to change the shirt to a blue color this is where

things get more complicated so even the word complicated is complicated haha I will make a selection of it change the

prompt to a blue shirt but somehow it's too strong for the option masked content original to change the color I tried to

increase the denoy strength a little but it still gave me the different variations of it we have an option that

works better with colors uh the fill option so I will try a few Generations uh with that and as you can see it's not

perfect but I get a hint of a blue shirt in there if I increase the denoise too much I will get a better shirt but it

will not fit the subject as well now I will drag this image to inpaint and then switch back to masked original because

now we have a blue color to work with you can also do this quicker in Photoshop by selecting and changing the

color quickly then using that on stable diffusion so it's easier to see the shapes and

colors to help with blending you can also try the uh option called soft and painting um select it and try some

generations with with it you can also expand it to access more options and settings I haven't played with all the

options because most of the time I managed to get what I wanted but you also have the help tab where you can

click and read more about what each option does if you found something helpful in this video please give it a

like thank you and have a great [Music] day