Dreambooth-Stable-Diffusion is a Dreambooth implementation based on Stable Diffusion, designed to fine-tune the model with a small number of images to learn a specific face, object, or style. This repository is maintained by Joe Penna, who modified the original repository with a particular focus on face training. The project provides extensive guides, including steps for running on Vast.ai, Google Colab, local PC (Windows and Ubuntu), and RunPod. Users can train via Jupyter notebook or the command line.
Main capabilities: support training faces, objects, and styles; support running on GPUs with 24GB VRAM (such as the 3090); provide configuration files and command-line parameter references; support training multiple subjects or concepts via captions; the trained model can be used to generate images of a specific subject or style; provide debugging suggestions.
Usage: Users need to clone the repository (git clone https://github.com/JoePenna/Dreambooth-Stable-Diffusion.git），安装依赖，准备训练图像和正则化图像，然后通过命令行或 notebook to run training. Training parameters include project_name, training_model, regularization_images, training_images, max_training_steps, class_word, token, etc. After generating the model, use the <token> <class> format in the prompt (for example, "joepenna person") to generate images.
Applicable scenarios: digital artists training their own characters and styles; filmmakers communicating with concept artists; individuals training their own portraits; comic designers, etc.
Limitations: requires a GPU with at least 24GB VRAM; training may be slow; this implementation does not fully implement Google's method of preserving latent space, which may cause all similar images to shift toward the trained subject; it is difficult to train two subjects consecutively; after training, the model file is about 11-12GB and needs to be pruned to about 2GB; training on others' works without authorization or using artists' names is discouraged.
