Mastering Stable Video Diffusion: A Step-by-Step Guide

today's tutorial is about stable video diffusion let me quickly show you what I did with

[Music] it I know that after seeing what Sor from open AI can do a lot of people lost

interest in stable video diffusion however we don't have access to that yet and it's not free I am using the stable

diffusion Forge UI SVD is already integrated you just need to look for the tab called SVD uh which stands for

stable video diffusion if you click on it you'll find a place where you can upload or drop your image if you scroll

down it says SVD checkpoint file name there you don't have a model yet so you have to download one there are different

sources but I downloaded one from Civ AI you can search for the name or access this link I use version 1.1 you can

download this into the following folder go to web UI then look for models the folder named SVD is where you place the

model now before moving on keep in mind that SVD needs a good video card with more than six to 8 gabt of video RAM if

you hit the refresh button you should be able to see it here and you can select it from the list now for limitations you

can only use videos with dimensions of 1,24 by 576 pixels or 576 by 1,24 pixels so upload an image with a similar ratio

for the rest of the settings use these for video frames 25 for motion bucket ID I use 127 is a parameter that controls

the level of motion in the generated video by adjusting this value you can influence how much motion is present a

higher motion bucket ID results in more pronounced and dynamic Motion in the video while a lower value leads to to a

calmer and more stable effect for fps6 copy the rest of the settings for all the remaining and for the sampler I used

ier a but you can experiment with others use the schedular Caris and for the seed you can change it to different numbers

until you find a variation that you like let me generate an image to show you how it works I'll use a prompt with a robot

just like you saw at the beginning of this video remember it only works on those exact sizes so use that for the

size either landscape or portrait optionally you can use an art style once I hit generate I'll get an

image like this cool robot under the image you'll see a button that lets you send the image to SVD click on it and

the image will appear of course you can also upload it manually here scroll down and look for the generate button click

on it and wait a minute or more depending on your video card's power after it's finished you can play the

result if you don't like like it or if there are a lot of Errors try again with a different seed until you get one that

looks okay don't expect to get perfect results on the first try there's a download button on the top right corner

so you can download it I'm not sure why they didn't put the video in the same folder by default it's put in the gradio

temp folder you can rightclick on the video and select copy video address then go to a folder and delete the end of the

path with the video and the beginning leaving you with the exact path where the video is saved as you can see the

video I generated appears here additionally in your normal export folder you have all the frames for this

video try different seeds and generate again until you get something that works the size and video quality are not so

good which is why I'm using a video upscaler like topaz video AI it's not cheap but it does a good job I'll show

you what I do just in case someone is interested in this process you simply browse or drag the video video here on

the right you can see the original size and FPS which is only six from the presets I choose upscale to 4K and

convert to 60fps then I hit export and wait for it to finish you'll see a green check mark

when it's done here is a side by-side comparison on the left is the original and on the right is the upscaled

version Additionally you can see it here with the screen split in half for a clearer comparison let's let me show you

another example of a cute cartoon snowman I'll use the same size that SVD supports um to get a better image in the

same size I'll lock the seed and enable the high resolution fix to get a bigger image that has fewer errors and looks a

little better optionally you can adjust it in Photoshop to have a clearer image and then send it or upload it to SVD I

scroll down and hit generate and while you weight if you look in the command window you can see more info on how much

memory is left from 24 gabes I have 18 gbt left so it seems like it's using around 6 gabt the first result is not

too bad it has some problems with the hands at the end you can download it and try another seed hoping for better

results sometimes you get lucky on the first try while other times it takes 5 to 10 tries to get a good one if it

still doesn't turn out how it should try a different image because it depends on the image as well its composition what's

seen in the image whether there's snow smoke fire Etc if you incorporate more elements in

the image there are more things that can move around to create more Dynamics but it can also lead to more mistakes so

play around with it and have fun here I picked one generation that looks okay and has fewer mistakes since the quality

is not so good I'm using topaz video AI to get this upscaled version uh I removed a few frames at the beginning

and at the end that had more obvious mistakes then I duplicated and reversed the video creating a loop that looks

kind of cool after that you can make it more interesting by adding some snow overlays on top and the result is quite

nice in the future models should produce better and better results so this is a good start I'd say here are a few more

examples I generated with it and upscaled afterward to get a smoother video um if you enjoyed this video

please leave a like thank you and have a great [Music]

[Music] day