Soup

Soup is a command-line application for large language model fine-tuning and post-training, with the core selling point of simplifying the training process: complete fine-tuning with a single YAML configuration and one command, without needing to SSH into a remote GPU machine and without dealing with complex configuration.

Main Uses

Soup is used for fine-tuning and post-training LLMs, supporting tasks including SFT, DPO, GRPO, KTO, ORPO, SimPO, IPO, BCO, RLHF, pretraining, MoE, long context, embedding, audio, and more. It provides configuration templates such as chat, code, tool-calling, medical, reasoning, vision, kto, orpo, simpo, ipo, bco, rlhf, pretrain, moe, longcontext, embedding, and audio.

Main Capabilities

• One command, one YAML: automatically handles batch size, GPU detection, and quantization.
• Layer Streaming: keeps the frozen base model in host memory and feeds it to the GPU layer by layer, thereby training an 8B model on a laptop GPU with 4 GB of VRAM. This feature is optional (stream_layers: true) and is still in BETA. The materials show that on an RTX 3050 Laptop 4 GB, with Llama-3.1-8B-Instruct + NF4, LoRA, batch 1, seq 512, the peak was 3.32 GB at 119.6 tok/s (952 MHz, v0.72.2 before the fix); after the fix there is another record of 208.6 tok/s and a 2.40 GB peak, but the materials explicitly state that the two cannot be directly compared because of different frequencies and memory modes.
• Disk tiering: layer streaming can read disk layers via a background thread using direct I/O, and stripe across NVMe drives via SOUP_LAYER_STREAM_STRIPE_DIRS. The materials record that on a 36 GB, 70B-scale, larger-than-memory NF4 store, the cold-start step dropped from 110.5 seconds with synchronous reads to 16.2–17.3 seconds, and with two drives from 17.4 seconds to 9.97 seconds (sustained-load probe, two rounds, 1.28x and 1.41x, no formal conclusion).
• Automatic data format recognition: Alpaca, ShareGPT, ChatML, preference pairs(DPO/ORPO/SimPO/IPO/KTO), vision, audio, ASR, plain text, embedding, RAFT, etc., can be automatically detected from JSONL, JSON, CSV, Parquet, or TXT.
• Model compatibility: any text generation model on HuggingFace Hub that can be loaded with AutoModelForCausalLM can be used, with no configuration changes needed. The materials list over 100 ready-made recipes including Llama 3.x/4, Qwen 2.5/3, Gemma 3, Mistral, Mixtral, DeepSeek R1/V3, Phi-4, and more.
• Command-line tools: soup init, soup train, soup data, soup doctor, soup adapters check, soup bench train, soup data clean, soup recipes verify, soup rewind, soup infer, soup ab, soup adapters sign, soup attest emit, soup ui, and more.
• Web UI: soup ui provides a local dashboard for experiments, training setup, live metrics, dataset exploration, and model chat.
• Export and deployment: supports targets such as ONNX, TensorRT, AWQ, GPTQ, BitNet, etc.; the materials also note that soup export --format awq and --format gptq will be removed in v0.77.

How to Use

1. Installation: it is recommended to use pipx or uv tool to create an isolated environment for Soup and add it to PATH: pipx install "soup-cli[train]" or uv tool install "soup-cli[train]". In a virtual environment, Colab, or Docker, you can directly pip install "soup-cli[train]". Note that double quotes must be used; single quotes are not usable in some shells. If you want to import soup_cli from code, you should use pip rather than pipx. Data/inspection-only commands can be installed lightweight; training requires the [train] extra.
2. Create a configuration: Run soup init to generate a template.
3. Training, testing, publishing: use commands such as soup train. See docs/commands.md for the full command list.
4. No local CUDA/PyTorch environment: you can use the image pushed to GHCR on each release.

Applicable scenarios

• Individual developers who only have a single 4 GB VRAM laptop GPU but want to fine-tune an 8B model.
• Teams that want to avoid SSH and infrastructure configuration and quickly start experiments.
• Users who need multiple post-training tasks (SFT, DPO, GRPO, KTO, ORPO, etc.) and multimodal data formats.
• Scenarios for local training (QLoRA) without the cloud.

Limitations and notes

• Only Python 3.10, 3.11, and 3.12 are supported; 3.13+ is not supported because the PyTorch stack has not yet been verified.
• CUDA GPU is recommended; Apple Silicon (MPS) is supported; CPU is experimental and extremely slow. For 7B model QLoRA, 8 GB+ VRAM is recommended.
• Tiered streaming is optional and still a BETA feature; the materials clearly state that its performance numbers are measurements under specific hardware and versions and cannot be directly extrapolated.
• Starting from v0.76.0, settings that were previously accepted but never took effect will be rejected at load time, such as training.quantization_aware: true, training.use_longlora, training.ebft_variant, training.gdpo_variant, etc.; before upgrading, it is recommended to run soup doctor --config soup.yaml first.
• Starting from v0.75, unknown configuration keys will be rejected at load time; v0.74 only reported them at load time.
• validation loss is now evaluated; grpo, ppo, and online_dpo no longer reserve validation rows by default (data.val_split is ignored).
• All non-standard grpo_variant values apply the grpo_beta KL penalty, and existing variant configurations cannot reproduce previous runs; grpo_beta: 0 means no KL and skips the reference forward pass.
• Telemetry is strictly opt-in (SOUP_TELEMETRY=1, off by default).
• The project is licensed under Apache-2.0, free; donations are processed by Stripe under the name MePlay, Inc.
• The materials do not provide specific fees, prices, or platform availability information.