SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI imageshunyuan-image3-finetune

hunyuan-image3-finetune

hunyuan-image3-finetune is a repository of training scripts developed by the PhotonAISG team for the Tencent HunyuanImage-3.0 model. HunyuanImage-3.0 is currently the largest open-source unified multimodal model (80B MoE), with both image understanding and generation capabilities, but the official repository only provides inference code. This project is based on Hugging Face Trainer, DeepSpeed, and PEFT, and provides a complete fine-tuning workflow, supporting pure-text language model fine-tuning and text-to-image generation training, as well as LoRA and QLoRA low-VRAM solutions. The repository includes data preparation tools, configuration files, and multi-GPU and Slurm cluster training scripts.

Open website

About this tool

Purpose

This repository provides fine-tuning training scripts for HunyuanImage-3.0. HunyuanImage-3.0 is an 80B MoE unified vision-language model that can perform both understanding and generation tasks, and the official repository contains only inference code. These scripts were independently developed by the Pixo team for research and educational purposes.

Main Capabilities

  • Two training modes
    • Pure-text fine-tuning: supervised fine-tuning based on the language model
    • Image generation training: flow-matching-based text-to-image generation
    • Note: The image understanding (MMU) pipeline has not yet been implemented.
  • Parameter-efficient fine-tuning
    • Supports LoRA fine-tuning
    • Supports QLoRA (4-bit / 8-bit quantization), enabling memory-efficient training on a single GPU
  • Training framework
    • Based on Hugging Face Transformers Trainer
    • Integrates DeepSpeed (including ZeRO-3 configuration)
    • Integrates PEFT, Diffusers
  • Data preparation tools
    • prepare_text_data.py: converts text formats such as Alpaca and ShareGPT into training JSONL
    • prepare_image_gen_data.py: converts an image directory and a CSV description into training JSONL, supporting instruct and pretrain styles
    • preprocess_images.py: pre-encodes images into latent representations using the VAE encoder to accelerate training
  • Training scripts
    • train_multimodal.py: main training script, supporting pure-text and image generation training
    • train.sh: pure-text fine-tuning example (2 GPU + DeepSpeed, default Alpaca dataset)
    • train_image.sh: image generation training example
    • train_image_qlora.sh: single-GPU memory-efficient image generation training (4-bit/8-bit)
    • train_slurm.sh / train_image_slurm.sh: Slurm cluster multi-node version
  • Advanced features
    • Controllable text and image loss weights
    • Flow matching weighting scheme:sigma_sqrt、logit_normal、mode、cosmap、none
    • Supports moe_drop_tokens to save VRAM at large resolutions

Usage

  1. Download the base model from the official repository
  2. Prepare training data (JSONL format, supporting pure text, instruct-style image generation, and pretrain-style image generation)
  3. Configure the training script
  4. Run training:
    # 纯文本微调
bash train.sh
# 图像生成训练
bash train_image.sh
# Slurm 集群
bash train_slurm.sh

Applicable Scenarios and Limitations

  • Suitable for community fine-tuning of HunyuanImage-3.0, style adapter training, and text-to-image generation customization.
  • Training requires relatively high VRAM; QLoRA can lower the barrier. VRAM requirements are based on tests with batch size 1 and short prompts (<50 tokens); long prompts or larger batches require more VRAM.
  • The image understanding pipeline has not yet been implemented.
  • The repository contains only training scripts; for inference code and documentation, please refer to the official repository.
  • Full model training requires significantly more VRAM and is not covered by the examples.
  • The official repository tokenizer_wrapper.py has a bug in instruct-style training, which this repository has fixed.
  • License: This repository is AGPL; the official Hunyuan Image 3.0 repository is under the Tencent Hunyuan Community License Agreement.

Related tools

Browse category