How to Fine-tune Stable Diffusion using Dreambooth (2024)

FAQs

How to fine tune Stable Diffusion with DreamBooth? ›

First, download the pre-trained Stable Diffusion model as a starting point. Then train this model with a few images of a subject. To achieve this, choose a non-word as an identifier, such as unqtkn . When fine-tuning the model with this subject, you teach the model that the prompt is A photo of a unqtkn <class> .

Now You Can Full Fine Tune / DreamBooth Stable Diffusion XL (SDXL) with only 10.3 GB VRAM via OneTrainer — Both U-NET and Text Encoder 1 is trained — Compared 14 GB config vs slower 10.3 GB Config.

You can use as few as 5 images, but 10-20 images is better. The more images you use, the better the fine-tune will be.

Windows systems require a minimum of 16GB of video memory, Linux systems require a minimum of 8GB of video memory.

You can play around with different Stable Diffusion models to get the best NSFW content if you have an excellent artistic sense. You have to choose the best-suited model for the type of NSFW images you want to create to achieve the desired output.

Class images: Denote the images generated using the "class prompt" for using prior preservation in DreamBooth training. We leverage the pre-trained model before fine-tuning it to generate these class images. Typically, 200 - 300 class images are enough.

Stable Diffusion requires a decent graphics card to run efficiently, but there are no specific requirements beyond that. However, it is recommended to have a dedicated graphics card with at least 2 GB of VRAM for optimal performance.

Because stable diffusion can be computationally intensive, most developers believe a GPU is required in order to run.

What is the best image size for Stable Diffusion? Stable Diffusion can create images from 64×64 to 1024×1024 pixels, but optimal results are achieved with its default 512×512 size. This size ensures consistency, diversity, speed, and manageable memory usage.

How does DreamBooth work? ›

The methodology used to run implementations of DreamBooth involves the fine-tuning the full UNet component of the diffusion model using a few images (usually 3--5) depicting a specific subject. Images are paired with text prompts that contain the name of the class the subject belongs to, plus a unique identifier.

Updating your Personal AI Profile requires half the number of final edits as the last AI Profile training but with a minimum of 3,000 edits. For example, if you created your Personal AI Profile with 3,000 photos, you will need 3,000 new photos to run an effective Fine-tune update.

For training objects or particular people, or a small dataset of images try a batch size of 2-4. This will make more of the gradients get calculated. Gradients are processed across the batch.

but 24gb of vram is a marketing ploy. Consoles this gen only have 16gb total, meaning more than 16gb of vram just isn't likely to be necessary for any games for the next few years. None of the cards today are likely to be all that great 6 or 7 years from now, anyway. 12~16gb for today is about right.

Stable Diffusion is ideal for generating general images but lacks personalization, requiring extensive training data. In contrast, DreamBooth is tailored for customization, demands a smaller dataset, and excels in generating images with specific subjects in various scenarios.

To upscale images using stable diffusion, begin by selecting the desired input image. Next, adjust the generation parameters, including the scale, batch size, and batch count, to suit your preferences. After setting the parameters, initiate the upscaling process by clicking on the generate button.

Under Extensions>Available, click on the “Load from:” button to show all the available extensions. Find the DreamBooth extension and click on "Install." Next, go to the “Installed” tab and click on the “Apply and restart UI” button. Your Web UI will restart, and you should be able to see the Dreambooth tab.

Here are the best practices for training a high-quality Stable Diffusion model: Curate High-Quality Training Data: The quality of your training data will have a significant impact on the quality of your model's output. You should make sure to use a large and diverse dataset of images that are relevant to your use case.