SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI imagesai-toolkit

ai-toolkit

AI Toolkit (Ostris AI Toolkit) is a free and open-source diffusion model training suite for fine-tuning image and video generation models. It offers both GUI and CLI usage, supports training the latest models on consumer-grade hardware, and is described by its author as an all-in-one training solution. The tool covers a large number of image, video, and editing models, including FLUX.1, FLUX.2, Stable Diffusion XL, SD 1.5, Qwen-Image, the Wan 2.1/2.2 series of video models, and LTX-2, while also supporting network types such as LoRA and LoKr as well as training specified layers. Installation can be done through the built-in experimental AI Toolkit Manager, which automatically detects hardware, configures the PyTorch environment, and obtains local Node.js and FFmpeg, or manually by following the manual. Training tasks can be defined through YAML configuration files, support resuming from checkpoints, and can be run on cloud platforms such as RunPod and Modal.

Open website

About this tool

Purpose

AI Toolkit is an open-source training suite for fine-tuning diffusion models, aimed at image and video generation models. The author positions it as an "all in one" training solution, intended to let users train the latest models on consumer-grade hardware, and it provides both GUI and CLI ways to run. The project is free and open-source.

Main capabilities

  • Supports a large number of mainstream image models, including FLUX.1, the FLUX.2 series, Flex.1/Flex.2, Chroma, Lumina2, the Qwen-Image series, HiDream I1, OmniGen2, the Z-Image series, SDXL, SD 1.5, ERNIE-Image, Nucleus-Image, Boogu Image, Ideogram 4 FP8, PRXPixel, Iris-3B, Anima, the Krea 2 series, Ming-Image 0.1 Design, Mage-Flow, and more.
  • Supports instruction/editing models, such as FLUX.1-Kontext-dev, the Qwen-Image-Edit series, HiDream E1, Boogu Image Edit, Krea 2 Edit training, Mage-Flow Edit, and more.
  • Supports video models, such as multiple sizes of Wan 2.1 and Wan 2.2 T2V/I2V/TI2V, Lightricks LTX-2/2.3/2.5, MiniMax-H3, MiniMax-H3 Ref2V.
  • Also lists audio-related models, such as Ace Step 1.5/XL, YuE2, Qwen2.5-Omni, and Zeta-Chroma.
  • Supports LoRA and LoKr network type training, and can specify or exclude specific layers/weight groups through network kwargs such as only_if_contains and ignore_if_contains (for example, training only FLUX.1's single_transformer).

Usage

Installation

The recommended approach is to use the repository's built-in experimental AI Toolkit Manager: it detects hardware, configures a suitable PyTorch build, creates a Python environment, and places local copies of Node.js and FFmpeg into the ai-toolkit folder, without performing system-level installation. On each startup it checks for and applies updates (local modifications will not be overwritten; if a file has been modified, the update is skipped and a warning is given), then at http://localhost:8675 launches the UI. The only requirement is git (on Windows the manager can obtain a portable version of git for updates, but cloning the repository still requires git to be installed). Start via platform scripts: Linux (x86_64 and ARM64, including DGX Spark / DGX OS), macOS (Apple Silicon, experimental), Windows (double-click run_windows.bat). The manager can also be used directly in the terminal, suitable for headless servers.

Manual installation requires python >=3.10 (3.12 recommended) and an Nvidia GPU with sufficient VRAM; the steps also apply to ARM64 Linux (including DGX Spark / DGX OS). If you encounter problems on Windows, you can use the third-party simple installation script https://github.com/Tavris1/AI-Toolkit-Easy-Install。

Interface and Tasks

The UI is the AI Toolkit's web interface, used to start, stop, and monitor tasks, and training can also be completed with a few clicks. The UI does not need to run continuously; it is only needed when starting, stopping, and monitoring tasks. A token can be set for the UI to prevent unauthorized access, making it suitable for running on an exposed server. When hosting in the cloud or on an insecure network, it is recommended to set the authentication password via the environment variable AI_TOOLKIT_AUTH.

Configuration and Training

Copy an example configuration (such as config/examples/train_lora_flux_24gb.yaml; for schnell use train_lora_flux_schnell_24gb.yaml) to the config folder and rename it to a custom .yml, modify it according to the comments, then run python run.py config/whatever_you_want.yml. After startup, a directory will be created according to the name and training folder in the configuration to store checkpoints and images. You can stop it with ctrl+c; when resuming, it will continue from the last checkpoint. Note that pressing ctrl+c during the saving process may corrupt the checkpoint, so you need to wait for saving to complete.

Cloud Platforms

GPUs can be rented from multiple cloud providers. RunPod's official template and getting-started video are provided in the materials; the Modal method requires pip install modal, modal setup authentication, obtaining a READ token and access to the Flux.1-dev model, logging in with huggingface-cli login and entering the token, uploading the dataset, copying the config/examples/modal example configuration and modifying it, editing run_modal.py to set the local ai-toolkit path and GPU/timeout, and running modal run run_modal.py --config-file-list-str=/root/ai-toolkit/config/whatever_you_want.yml. Models/samples/optimizers are stored in Storage > flux-lora-models, and you can view and download them with modal volume ls and modal volume get.

Dataset Preparation

The dataset is generallya folder containing images and corresponding text files. Currently only jpg, jpeg, and png are supported (WebP has issues). The text file has the same name as the image but with the .txt extension, and its content is only the caption. You can add [trigger] to the caption; if trigger_word is configured, it will be replaced automatically. Images will not be enlarged; they will be reduced and placed into buckets for batching. There is no need to crop or resize them yourself; the loader will handle it automatically and supports different aspect ratios.

Applicable Scenarios

Suitable for users who want to fine-tune diffusion models (image, video, editing) on local or cloud GPUs, including scenarios that require LoRA/LoKr, specified layer training, simplifying operations through a GUI, or fine-grained control of the training process through CLI/YAML.

Limitations and Notes

  • Manager is still experimental and may have issues.
  • Manual installation requires an Nvidia GPU and sufficient VRAM; macOS Apple Silicon support is experimental.
  • The dataset only supports jpg/jpeg/png, and WebP has issues.
  • Interrupting while saving a checkpoint may cause corruption.
  • The author requests that issues only be submitted when there is a bug in the code; for general questions, it is recommended to ask on Discord, not by private message.
  • When hosting the UI in the cloud or on an insecure network, an authentication token must be set.

Related tools

Browse category