SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI imagesflymyai-lora-trainer

flymyai-lora-trainer

FlyMyAI LoRA Trainer is an open-source LoRA training implementation by FlyMy.AI, used to train LoRA (Low-Rank Adaptation) layers for the Qwen-Image, Qwen-Image-Edit, and FLUX.1-dev models. It is based on Hugging Face diffusers, supports configuring training parameters via YAML, provides dataset preparation and validation tools, and includes pretrained LoRA weight examples and ComfyUI workflows. The project is in the development and refinement stage, but it is already usable for training.

Open website

About this tool

FlyMyAI LoRA Trainer is an open-source LoRA training implementation by FlyMy.AI, used to train LoRA (Low-Rank Adaptation) layers for the Qwen-Image, Qwen-Image-Edit, and FLUX.1-dev models. FlyMy.AI positions itself as B2B infrastructure for generative AI, used to build and run GenAI media agents.

Purpose The main purpose of this project is to enable users to perform LoRA fine-tuning on text-to-image models such as Qwen-Image, Qwen-Image-Edit, and FLUX.1-dev for specific characters, styles, or image editing tasks. It supports control-based image editing training and is compatible with Hugging Face diffusers.

Main capabilities

  • Supports LoRA training for the Qwen-Image, Qwen-Image-Edit, and FLUX.1-dev models.
  • Provides full training support for Qwen-Image and Qwen-Image-Edit.
  • Supports control-based image editing LoRA training.
  • Provides high-quality portrait and character training for FLUX.
  • Configuration via YAML files.
  • Freezes model weights during training.
  • Provides dataset preparation instructions and a dataset validation script (utils/validate_dataset.py).
  • Provides pretrained LoRA weight examples (Qwen LoRA and FLUX LoRA).
  • Provides ComfyUI workflows suitable for LoRA models after Qwen and FLUX training.
  • Supports a workflow for training on GPUs with less than 24GiB of VRAM.

Usage

  1. Clone the repository and enter the directory: git clone https://github.com/FlyMyAI/flymyai-lora-trainer cd flymyai-lora-trainer
  2. Install dependencies: pip install -r requirements.txt
  3. Install the latest diffusers from GitHub: pip install git+https://github.com/huggingface/diffusers
  4. Optionally download pretrained LoRA weights:

    Qwen LoRA weights

    git clone https://huggingface.co/flymy-ai/qwen-image-realism-lora

    FLUX LoRA weights

    git clone https://huggingface.co/flymy-ai/flux-dev-anne-hathaway-lora

    Or download specific files

    wget https://huggingface.co/flymy-ai/qwen-image-realism-lora/resolve/main/flymy_realism.safetensors wget https://huggingface.co/flymy-ai/flux-dev-anne-hathaway-lora/resolve/main/pytorch_lora_weights.safetensors
  5. Prepare the dataset: Training data must follow the same format, with each image corresponding to a text file of the same name. Images support PNG, JPG, JPEG, and WEBP formats. For Qwen-Image-Edit training, target images/descriptions and control images must be stored in separate directories. It is recommended to use high-resolution images (1024x1024 or higher is recommended), with detailed and accurate descriptions, and a dataset of at least 10-50 image-text pairs. FLUX character training can use the trigger word "ohwx woman" or "ohwx man"; Qwen training does not require a specific trigger word. Florence-2 can be used to automatically generate descriptions.
  6. Data validation: Run the included validation tool to check the correspondence between images and text files and the naming conventions.
  7. Start training:
    • Qwen-Image LoRA training: Use a configuration file (such as train_lora.yaml) to run the training script.
    • Qwen-Image full training: Use a configuration file (such as train_full_qwen_image.yaml) to run; this method was tested in an NVIDIA H200 GPU environment.
    • Qwen-Image-Edit LoRA training: Use a dedicated training script; the configuration file must include parameters such as img_dir and control_dir.
    • Qwen-Image-Edit full training: Use a configuration file (such as train_full_qwen_edit.yaml) to run.
    • FLUX.1-dev LoRA training: Use a configuration file (such as train_flux_config.yaml) to run.
  8. Inference and generation: After training, the model can be loaded from the checkpoint directory for inference. The project provides loading and generation examples for Qwen-Image, Qwen-Image-Edit, and FLUX LoRA. FLUX generation requires the trigger word "ohwx woman".
  9. ComfyUI usage: Download the latest ComfyUI, install dependencies, download the model weights and place them in the corresponding folders under ComfyUI/models/, download the pretrained LoRA weights and place them in ComfyUI/models/loras/, then load the workflow file (use qwen_image_lora_example.json for Qwen, and flux_anne_hathaway_lora_example.json for FLUX).

Applicable scenarios

  • Train FLUX LoRA for a specific person or character to generate high-quality portraits.
  • Train style or subject LoRA for Qwen-Image.
  • Train Qwen-Image-Edit LoRA for control-based image editing tasks.
  • Perform LoRA training in a local GPU environment (including less than 24GiB VRAM).
  • Use the trained LoRA model for image generation through ComfyUI workflows.

Limitations

  • The project status is under development, with code being improved and test coverage being added; it is in the refinement stage but already usable.
  • Performance optimization is still ongoing, and test coverage is coming soon.
  • The method for full Qwen-Image training was tested in an NVIDIA H200 GPU environment; other environments are not explicitly stated.
  • The dataset size is recommended to be at least 10-50 image-text pairs; the specific results may vary depending on data quality.
  • The specific fees, prices, and platform details of some features (such as the online training platform) are not yet provided.

Related tools

Browse category